BatchJobsParam-class: Enable parallelization on batch systems

In BiocParallel: Bioconductor facilities for parallel evaluation

Description

This class is used to parameterize scheduler options on managed high-performance computing clusters.

Usage

BatchJobsParam(workers, cleanup = TRUE, 
    work.dir = getwd(), stop.on.error = TRUE, seed = NULL,
    resources = NULL, conffile = NULL, cluster.functions = NULL, 
    progressbar = TRUE, jobname = "BPJOB",
    reg.pars=list(seed=seed, work.dir=work.dir),
    conf.pars=list(conffile=conffile, cluster.functions=cluster.functions),
    submit.pars=list(resources=resources),
    ...)

Arguments

workers

integer(1)

Number of workers to divide tasks (e.g., elements in the first argument of bplapply) between. On Multicore and SSH backends, this defaults to all available nodes. On managed (e.g., slurm, SGE) clusters workers defaults to NA, meaning that the number of workers equals the number of tasks. See argument n.chunks in chunk and submitJobs for more information.

cleanup

logical(1)

BatchJobs creates temporary directories in the work.dir. If cleanup is set to TRUE (default), the directories are removed from the file systems automatically. Set this to FALSE whenever it might become necessary to utilize any special functionality provided by BatchJobs. To retrieve the registry, call loadRegistry on the temporary directory.

work.dir

character(1)

Directory to store temporary files. Note that this must be shared across computational nodes if you use a distributed computing backend. Default ist the current working directory of R, see getwd. Ignored when reg.pars is provided.

stop.on.error

logical(1)

Stop all jobs as soon as one jobs fails (stop.on.error == TRUE) or wait for all jobs to terminate. Default is TRUE.

seed	integer(1L)

Set an initial seed for the RNG. See makeRegistry for more information. Default is NULL where a random seed is chosen upon initialization. Ignored when reg.pars is provided.

resources

list()

List of job specific resources passed to submitJobs. Default is NULL where the resources defined in the configuration are used. Ignored when submit.pars is provided.

conffile

character(1)

URI to a custom BatchJobs configuration file used for execution. Default is NULL which relies on BatchJobs to handle configuration files. Ignored when conf.pars is provided.

cluster.functions

ClusterFunctions

Specify a specific cluster backend using on of the constructors provided by BatchJobs, see ClusterFunctions. Default is NULL where the default cluster functions defined in the configuration are used. Ignored when conf.pars is provided.

progressbar

logical(1)

Suppress the progress bar used in BatchJobs and be less verbose. Default is FALSE.

jobname

character(1)

Job name that is prepended to the output log and result files. Default is "BPJOB".

reg.pars

list()

List of parameters passed to BatchJobs::makeRegistry(). When present, user-supplied arguments seed and work.dir to BatchJobsParam are ignored.

conf.pars

list()

List of parameters passed to BatchJobs::setConfig(). When present, user-supplied arguments conffile, cluster.functions to BatchJobsParam are ignored.

submit.pars

list()

List of parameters passed to BatchJobs::submitJobs. When present, user-supplied argument resources to BatchJobsParam is ignored. submitJobs parameters reg, id cannot be set.

...	Addition arguments, currently not handled.

BatchJobsParam constructor

Return an object with specified values. The object may be saved to disk or reused within a session.

Methods

The following generics are implemented and perform as documented on the corresponding help page: bpworkers, bpnworkers, bpstart, bpstop, bpisup, bpbackend, bpbackend<-

Author(s)

Michel Lang, mailto:[e-mail]

See Also

getClass("BiocParallelParam") for additional parameter classes.

register for registering parameter classes for use in parallel evaluation.

Examples

p <- BatchJobsParam(progressbar=FALSE)
bplapply(1:10, sqrt, BPPARAM=p)

## Not run: 
## see vignette for additional explanation
funs <- makeClusterFunctionsSLURM("~/slurm.tmpl")
param <- BatchJobsParam(4, cluster.functions=funs)
register(param)
bplapply(1:10, function(i) sqrt)

## End(Not run)

BiocParallel documentation built on Nov. 8, 2020, 5:46 p.m.