mclapply2: Parallel Versions of 'lapply' with cores and memory control

Description Usage Arguments Value See Also

Description

mclapply2 is a mclapply modification from parallel package, to avoid a memory overload. The maximum number of cores is computed depending on the amount of memory used by the parent process and the amount of free memory available on the machine. Note: This number is under-estimated.

Usage

1
2
3
mclapply2(X, FUN, ..., mc.preschedule = TRUE, mc.set.seed = TRUE,
          mc.silent = FALSE, mc.cores = getOption("mc.cores", 2L),
          mc.cleanup = TRUE, mc.allow.recursive = TRUE)

Arguments

X

a vector (atomic or list) or an expressions vector. Other objects (including classed objects) will be coerced by as.list.

FUN

the function to be applied to (mclapply) each element of X or (mcmapply) in parallel to ....

...

For mclapply, optional arguments to FUN.

mc.preschedule

if set to TRUE then the computation is first divided to (at most) as many jobs are there are cores and then the jobs are started, each job possibly covering more than one value. If set to FALSE then one job is forked for each value of X. The former is better for short computations or large number of values in X, the latter is better for jobs that have high variance of completion time and not too many values of X compared to mc.cores.

mc.set.seed

See mcparallel.

mc.silent

if set to TRUE then all output on ‘stdout’ will be suppressed for all parallel processes forked (‘stderr’ is not affected).

mc.cores

The number of cores to use, i.e. at most how many child processes will be run simultaneously. The option is initialized from environment variable MC_CORES if set. Must be at least one, and parallelization requires at least two cores.

mc.cleanup

if set to TRUE then all children that have been forked by this function will be killed (by sending SIGTERM) before this function returns. Under normal circumstances mclapply waits for the children to deliver results, so this option usually has only effect when mclapply is interrupted. If set to FALSE then child processes are collected, but not forcefully terminated. As a special case this argument can be set to the number of the signal that should be used to kill the children instead of SIGTERM.

mc.allow.recursive

Unless true, calling mclapply in a child process will use the child and not fork again.

Value

A list of the same length as X and named by X.

See Also

mclapply from package parallel.


mcanouil/myScriptsMickael documentation built on May 22, 2019, 12:59 p.m.