rasterEngine: Engine for performing fast, easy-to-develop pixel and focal...

Description Usage Arguments Details Author(s) See Also Examples

View source: R/rasterEngine.R

Description

Engine for performing fast, easy-to-develop pixel and focal raster calculations with parallel processing capability.

Usage

1
2
3
4
5
6
7
8
rasterEngine(x, fun = NULL, args = NULL, window_dims = c(1, 1),
  window_center = c(ceiling(window_dims[1]/2), ceiling(window_dims[2]/2)),
  filename = NULL, overwrite = FALSE, outformat = "raster",
  additional_header = "ENVI", datatype = "FLT8S", processing_unit = NULL,
  chunk_format = "array", minblocks = "max", blocksize = NULL,
  outbands = NULL, outfiles = NULL, setMinMax = FALSE,
  compileFunction = FALSE, debugmode = FALSE, .packages = NULL,
  clearworkers = TRUE, verbose = FALSE, ...)

Arguments

x

Raster*. A Raster* or a list of named Raster* objects used as the input into the function. This is optional, as long as some Raster* was defined in "..."

fun

Function. A focal function to be applied to the image. See Details.

args

List. Arguments to pass to the function (see ?mapply). Note that the 'fun' should explicitly name the variables.

window_dims

Vector. The size of a processing window in col x row order. Be default, a single pixel (c(1,1).

window_center

Vector. The local coordinate of the center of a processing window. By default the middle of the processing window. CURRENTLY UNSUPPORTED.

filename

Character. Filename(s) of the output raster.

overwrite

Logical. Allow files to be overwritten? Default is FALSE.

outformat

Character. Outformat of the raster. Must be a format usable by hdr(). Default is 'raster'. CURRENTLY UNSUPPORTED.

additional_header

Character. Create additional output headers for use with other GIS systems (see hdr). Set to NULL to suppress. Default is "ENVI".

datatype

Character. Output number type. See ?dataType. Default is "FLT8S".

processing_unit

Character. ("single"|"chunk") Will be auto-set if not specified ("chunk" for pixel-processing, "single" for focal processing). See Details.

chunk_format

Character. The format to send the chunk to the function. Can be "array" (default), "raster", or "data.frame".

minblocks

Numeric. The minimum number of chunks to divide the raster into for processing. Defaults to 1.

blocksize

Numeric. The size (in rows) for a block of data. If unset, rasterEngine will attempt to figure out an optimal blocksize.

outbands

Numeric. If known, how many bands in each output file? Assigning this and outfiles will allow focal_hpc to skip the pre-check.

outfiles

Numeric. If known, how many output files? Assigning this and outbands will allow focal_hpc to skip the pre-check.

setMinMax

Logical. Run a setMinMax() on each output file after processing (this will slow the processing down). Default is FALSE.

compileFunction

Logical. Runs a byte-code compilation on the user function before running. Setting this to TRUE may speed up the execution. Default is FALSE.

debugmode

Logical. If TRUE, the function will enter debug mode during the test phase. Note the inputs will be an array of size 2 columns, 1 row, and how ever many input bands.

.packages

Character. A character vector of package names needed by the function (parallel mode only).

clearworkers

Logical. Force the workers to clear all objects upon completing (releasing memory)? Default=TRUE.

verbose

Logical. Enable verbose execution? Default is FALSE.

...

Raster*s. Named variables pointing to Raster* objects. See Details.

Details

rasterEngine is designed to execute a function on one or multiple Raster* object(s) using foreach, to achieve parallel reads, executions and writes. Parallel random writes are achieved through the use of mmap, so individual image chunks can finish and write their outputs without having to wait for all nodes in the cluster to finish and then perform sequential writing. On Windows systems, random writes are possible but apparently not parallel writes. rasterEngine solves this by trying to write to a portion of the image file, and if it finds an error (a race condition occurs), it will simply retry the writes until it successfully finishes. On Unix-alikes, truly parallel writes should be possible.

rasterEngine is a convienence wrapper for focal_hpc and, in general, should be used instead of focal_hpc directly.

rasterEngine operates in two modes, which have different input and outputs to the function:

Pixel based processing:

1) If chunk_format=="array" (default), the input to the function should assume an array of dimensions x,y,z where x = the number of columns in a chunk, y = the number of rows in the chunk, and z = the number of bands in the chunk. If chunk_format=="data.frame", the function receives a data.frame where each column of the data.frame is a single layer from the input raster(s). If chunk_format=="raster", the input to the function will be a raster subset.

Note that we are ordering the array using standards for geographic data, (columns, rows, bands), not how R usually thinks of arrays (rows, columns, bands).

2) The output of the function can be as follows: - A data.frame where each column is assumed to be a single layer in the output raster - An array with the x and y dimensions matching the input, and an arbitrary number of band outputs. Remember to order the dimensions as columns, rows, bands (x,y,z). - A numeric vector, where the output is assumed to be a single layer (similar to calc() ).

Local window processing:

1) The function should be written to process a SINGLE window at a time, given the dimensions of window_dims, so the input to the function should assume a window of dimensions window_dims with a local center defined by window_center. As with before, the input can be passed to the function as an array (suggested) or a small raster.

2) The output should be a single pixel value, so can either be a single value, or a vector (which is assumed to be multiple bands of a single pixel).

The speed of the execution when running in parallel will vary based on the specific setup, and may, indeed, be slower than a sequential execution (e.g. with calc() ), particularly on smaller files. Note that by simply running sfQuickStop(), rasterEngine will run in sequential mode.

Author(s)

Jonathan A. Greenberg ([email protected])

See Also

focal_hpc, foreach, mmap, dataType, hdr

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
library("raster")
# Pixel-based processing on one band:
apply_multiplier <- function(inraster,multiplier)
{
# Note that inraster is received by this function as a 3-d array (col,row,band)
multiplied_raster <- inraster * multiplier
return(multiplied_raster)
}

tahoe_lidar_highesthit <-
setMinMax(raster(system.file(
"external/tahoe_lidar_highesthit.tif", package="spatial.tools")))

# Note that you can use any parallel backend that can be registered with foreach.
# sfQuickInit() will spawn a PSOCK cluster using the parallel package.
# sfQuickInit(cpus=2)
tahoe_lidar_highesthit_multiplied <- rasterEngine(
inraster=tahoe_lidar_highesthit,
fun=apply_multiplier,
args=list(multiplier=3.28084))
# sfQuickStop()
 
## Not run:  
# Pixel-based processing on more than one band: 
ndvi <- function(GRNIR_image)
{
# The input array will have dim(GRNIR_image)[3] equal
# to 3, because the input image has three bands.
# Note: the following two lines return an array,
# so we don't need to manually set the dim(ndvi) at the
# end.  If we didn't use drop=FALSE, we'd need to
# coerce the output into a 3-d array before returning it.
red_band <- GRNIR_image[,,2,drop=FALSE]
nir_band <- GRNIR_image[,,3,drop=FALSE]
ndvi <- (nir_band - red_band)/(nir_band + red_band)
# The following is not needed in this case:
# dim(ndvi) <- c(dim(GRNIR_image)[1],dim(GRNIR_image)[2],1)
return(ndvi)
}

tahoe_highrez <- setMinMax(
brick(system.file("external/tahoe_highrez.tif", package="spatial.tools")))

sfQuickInit(cpus=2)
tahoe_ndvi <- rasterEngine(GRNIR_image=tahoe_highrez,fun=ndvi)
sfQuickStop()

# Focal-based processing:
mean_smoother <- function(inraster)
{
smoothed <- apply(inraster,3,mean)
return(smoothed)
}

# Apply the function to a 3x3 window:
sfQuickInit(cpus=2)
tahoe_3x3_smoothed <- rasterEngine(inraster=tahoe_highrez,fun=mean_smoother,window_dims=c(3,3))
sfQuickStop()

# Example with 7 x 7 window in full parallel mode:
sfQuickInit()
tahoe_7x7_smoothed <- rasterEngine(inraster=tahoe_highrez,fun=mean_smoother,window_dims=c(7,7))
sfQuickStop()

## End(Not run)

spatial.tools documentation built on April 23, 2018, 3 a.m.