processNlData: Downloads nightlight tiles and country polygons and calls the...
In chrisvwn/Rnightlights: Satellite Nightlight Data Extraction

Description Usage Arguments Details Value Examples

Downloads nightlight tiles and country polygons in preparation for the appropriate functions to process them. Given a list of countries and nlPeriods and an nlType, processNlData will first determine which countries and periods do not actually have data i.e. have not been previously processed. From the list of countries and prediods that need processing, it determines which nightlight tiles require to be downloaded. At the same time, it downloads any country polygons which have not already been downloaded.

processNlData(
  ctryCodes,
  admLevels,
  nlTypes,
  configNames,
  extensions,
  multiTileStrategy = pkgOptions("multiTileStrategy"),
  multiTileMergeFun = pkgOptions("multiTileMergeFun"),
  removeGasFlaresMethod = pkgOptions(paste0("removeGasFlaresMethod_", nlType)),
  nlPeriods,
  nlStats = pkgOptions("nlStats"),
  custPolyPath = NULL,
  gadmVersion = pkgOptions("gadmVersion"),
  gadmPolyType = pkgOptions("gadmPolyType"),
  downloadMethod = pkgOptions("downloadMethod"),
  cropMaskMethod = pkgOptions("cropMaskMethod"),
  extractMethod = pkgOptions("extractMethod")
)

`ctryCodes`	(character) the list of countries to be processed
`admLevels`	(character) the list of admin levels in the given countries at which to calculate stats
`nlTypes`	(character) the types of nightlights to process
`configNames`	character the config shortnames of rasters being processed
`extensions`	character the extensions of rasters being processed
`multiTileStrategy`	(character) How to handle multiple tiles per nlPeriod
`multiTileMergeFun`	(character) The function to use to merge tiles
`removeGasFlaresMethod`	character The method to use to perform gas flare removal or NULL to disable
`nlPeriods`	(character) the nlPeriods of interest
`nlStats`	(character) the statistics to calculate. If not provided will calculate the stats specified in `pkgOptions("nlStats")`
`custPolyPath`	(character) Alternative to GADM. A path to a custom shapefile zip
`gadmVersion`	(character) The GADM version to use
`gadmPolyType`	(character) The format of polygons to download from GADM
`downloadMethod`	(character) The method used to download rasters and polygons
`cropMaskMethod`	(character) The method used to crop and mask satellite rasters
`extractMethod`	(character) The method used to extract and perform functions on raster data

processNlData then calls processNlCountry with the nlType supplied as a parameter. The processing is essentially the same for all nlTypes.

This is the main entry-point to the package and the main user-facing function. However, it works in batch mode and caches all data without returning any data to the user. However, it will return TRUE/FALSE depending on whether it completed successfully. Since it saves state regularly it can be run multiply in case of failure until it finally returns TRUE. This is where the only constraints are downloading and processing errors due to bandwidth or other resource constraints. A good example is running a long-running batch on an AWS EC2 spot-priced machine where the server may be decommissioned at any time. See more in the examples.

None

#long running examples which may require large downloads
## Not run: 
#Example 1: process monthly VIIRS nightlights for all countries at the
    lowest admin level and for all nlPeriods available e.g. to create a
    local cache or repo

    processNlData() #process VIIRS nightlights for all countries and all periods

#Example 2: process monthly VIIRS nightlights for all countries in 2014 only

    nlPeriods <- getAllNlPeriods("VIIRS.M") #get a list of all nightlight periods to present-day

    nlPeriods <- nlPeriods[grep("^2014", nlPeriods)] #filter only periods in 2014

    processNlData(nlTypes="VIIRS.M", nlPeriods=nlPeriods)

#Example 3: process OLS nightlights for countries KEN & RWA from 1992
#     to 2000

    cCodes <- c("KEN", "RWA")

    nlPeriods <- getAllNlPeriods("VIIRS.M")

    nlPeriods <- nlRange("1992", "2000", "OLS.Y")

    processNlData(ctryCodes=cCodes, nlPeriods=nlPeriods)

#Example 4: process VIIRS nightlights for countries KEN & RWA in 2014 Oct to 2014 Dec only

    processNlData(ctryCodes=c("KEN", "RWA"), nlTypes="VIIRS.M",
        nlPeriods=c("201410", "201411", "201412"))

#Example 5: process all nightlights, all countries, all stats in one thread

   processNlData()

#Example 6: process all VIIRS monthly nightlights, all countries, all stats with each
#   year in a separate thread. Create a separate R script for each year as follows:

    library(Rnightlights)

    nlPeriods <- getAllNlPeriods("VIIRS.M")

    nlPeriods_2014 <- nlPeriods[grep("^2014", nlPeriods)]

    processNlData(nlPeriods=nlPeriods_2014)

    #Run the script from the command line as:

    #R CMD BATCH script_name_2014.R
    
## End(Not run)