bploop: Internal Functions for SNOW-style Parallel Evaluation

In Bioconductor/BiocParallel: Bioconductor facilities for parallel evaluation

Internal Functions for SNOW-style Parallel Evaluation

Description

The functions documented on this page are primarily for use within BiocParallel to enable SNOW-style parallel evaluation, using communication between manager and worker nodes through sockets.

Usage

## S3 method for class 'lapply'
bploop(manager, X, FUN, ARGS, BPPARAM, BPOPTIONS = bpoptions(), BPREDO, ...)

## S3 method for class 'iterate'
bploop(manager, ITER, FUN, ARGS, BPPARAM, BPOPTIONS = bpoptions(),
       REDUCE, BPREDO, init, reduce.in.order, ...)

Arguments

manager	An object representing the manager node. For workers, this is the node to which the worker will communicate. For managers, this is the form of iteration – lapply or iterate.
X	A vector of jobs to be performed.
FUN	A function to apply to each job.
ARGS	A list of arguments to be passed to FUN.
BPPARAM	An instance of a BiocParallelParam class.
ITER	A function used to generate jobs. No more jobs are available when ITER() returns NULL.
REDUCE	(Optional) A function combining two values returned by FUN into a single value.
init	(Optional) Initial value for reduction.
reduce.in.order	(Optional) logical(1) indicating that reduction must occur in the order jobs are dispatched (TRUE) or that reduction can occur in the order jobs are completed (FALSE).
BPREDO	(Optional) A list of output from bplapply or bpiterate with one or more failed elements.
...	Additional arguments, ignored in all cases.
BPOPTIONS	Additional options to control the behavior of the parallel evaluation, see bpoptions.

Details

Workers enter a loop. They wait to receive a message (R list) from the manager. The message contains a type element, with evaluation as follows:

“EXEC”

Execute the R code in the message, returning the result to the manager.

“DONE”

Signal termination to the manager, terminate the worker.

Managers under lapply dispatch pre-determined jobs, X, to workers, collecting the results from and dispatching new jobs to the first available worker. The manager returns a list of results, in a one-to-one correspondence with the order of jobs supplied, when all jobs have been evaluated.

Managers under iterate dispatch an undetermined number of jobs to workers, collecting previous jobs from and dispatching new jobs to the first available worker. Dispatch continues until available jobs are exhausted. The return value is by default a list of results in a one-to-one correspondence with the order of jobs supplied. The return value is influenced by REDUCE, init, and reduce.in.order.

Author(s)

Valerie Obenchain, Martin Morgan. Derived from similar functionality in the snow and parallel packages.

Examples

## These functions are not meant to be called by the end user.

Bioconductor/BiocParallel documentation built on Oct. 31, 2024, 6:58 a.m.