batchReduceResults: Reduces results via a binary function and adds jobs for this...

Description Usage Arguments Value Examples

Description

Each jobs reduces a certain number of results on one slave. You can then submit these jobs to the batch system. Later, you can do a final reduction with reduceResults on the master.

Usage

1
2
batchReduceResults(reg, reg2, fun, ids, part = NA_character_, init,
  block.size, more.args = list())

Arguments

reg

[Registry]
Registry whose results should be reduced by fun.

reg2

[Registry]
Empty registry that should store the job for the mapping.

fun

[function(aggr, job, res, ...)]
Function to reduce results with.

ids

[integer]
Ids of jobs whose results should be reduced with fun. Default is all jobs.

part

[character] Only useful for multiple result files, then defines which result file part(s) should be loaded. NA means all parts are loaded, which is the default.

init

[any]
Initial object for reducing.

block.size

[integer(1)]
Number of results reduced in one job.

more.args

[list]
A list of other arguments passed to fun. Default is empty list.

Value

Vector of type integer with job ids.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# generating example results:
reg1 = makeRegistry(id = "BatchJobsExample1", file.dir = tempfile(), seed = 123)
f = function(x) x^2
batchMap(reg1, f, 1:20)
submitJobs(reg1)
waitForJobs(reg1)

# define function to reduce on slave, we want to sum the squares
myreduce = function(aggr, job, res) aggr + res

# sum 5 results on each slave process, i.e. 4 jobs
reg2 = makeRegistry(id = "BatchJobsExample2", file.dir = tempfile(), seed = 123)
batchReduceResults(reg1, reg2, fun = myreduce, init = 0, block.size = 5)
submitJobs(reg2)
waitForJobs(reg2)

# now reduce one final time on master
reduceResults(reg2, fun = myreduce)

BatchJobs documentation built on May 15, 2019, 1:05 a.m.