batchReduceResults: Reduces results via a binary function and adds jobs for this...

In BatchJobs: Batch Computing with R

Reduces results via a binary function and adds jobs for this to a registry.

Description

Each jobs reduces a certain number of results on one slave. You can then submit these jobs to the batch system. Later, you can do a final reduction with reduceResults on the master.

Usage

batchReduceResults(
  reg,
  reg2,
  fun,
  ids,
  part = NA_character_,
  init,
  block.size,
  more.args = list()
)

Arguments

reg	[Registry] Registry whose results should be reduced by fun.
reg2	[Registry] Empty registry that should store the job for the mapping.
fun	[function(aggr, job, res, ...)] Function to reduce results with.
ids	[integer] Ids of jobs whose results should be reduced with fun. Default is all jobs.
part	[character] Only useful for multiple result files, then defines which result file part(s) should be loaded. NA means all parts are loaded, which is the default.
init	[any] Initial object for reducing.
block.size	[integer(1)] Number of results reduced in one job.
more.args	[list] A list of other arguments passed to fun. Default is empty list.

Value

Vector of type integer with job ids.

Examples

# generating example results:
reg1 = makeRegistry(id = "BatchJobsExample1", file.dir = tempfile(), seed = 123)
f = function(x) x^2
batchMap(reg1, f, 1:20)
submitJobs(reg1)
waitForJobs(reg1)

# define function to reduce on slave, we want to sum the squares
myreduce = function(aggr, job, res) aggr + res

# sum 5 results on each slave process, i.e. 4 jobs
reg2 = makeRegistry(id = "BatchJobsExample2", file.dir = tempfile(), seed = 123)
batchReduceResults(reg1, reg2, fun = myreduce, init = 0, block.size = 5)
submitJobs(reg2)
waitForJobs(reg2)

# now reduce one final time on master
reduceResults(reg2, fun = myreduce)

BatchJobs documentation built on March 21, 2022, 5:05 p.m.