do_batch: Split a repetitive task into smaller batches.

In artichaud1/dobatch: Split a repetitive task into batches, saving intermediate results to disk.

Description

This can be useful if the task risks failure and you don't want to restart from scratch, or if you want to be able to stop a long task midway through.

Usage

do_batch(f, args = NULL, batch_size = NULL, out_folder = ".",
  file_prefix = "batch_", .combine = bind_rows, verbose = TRUE, ...)

Arguments

f	The function you want to call.
args	List of arguments passed to f. Each argument can be a list or vector.
batch_size	Size of the batches.
out_folder	Folder where to store the intermediate results (string).
file_prefix	Prefix for the intermediate result files (string).
.combine	Function used to combine the results.
verbose	Print progress (boolean).

Details

Given a function and a list of argument vectors, split the argument vectors into batches of fixed size, then call the function with its arguments, saving the results to disk at the end of each batch.

Value

The final aggregate result of the operation.

artichaud1/dobatch documentation built on May 21, 2019, 9:34 a.m.