parallel: Evaluate an expression asynchronously in a separate process

In jonclayden/multicore: Parallel processing of R code on machines with multiple cores or CPUs

Description

parallel starts a parallel process which evaluates the given expression.

mcparallel is a synonym for parallel that can be used at top level if parallel is masked by other packages. It should not be used in other packages since it's just a shortcut for importing multicore::parallel.

collect collects results from parallel processes.

Usage

parallel(expr, name, mc.set.seed = FALSE, silent = FALSE)
mcparallel(expr, name, mc.set.seed = FALSE, silent = FALSE)
collect(jobs, wait = TRUE, timeout = 0, intermediate = FALSE)

Arguments

expr	expression to evaluate (do not use any on-screen devices or GUI elements in this code)
name	an optional name (character vector of length one) that can be associated with the job.
mc.set.seed	if set to TRUE then the random number generator is seeded such that it is different from any other process. Otherwise it will be the same as in the current R session.
silent	if set to TRUE then all output on stdout will be suppressed (stderr is not affected).
jobs	list of jobs (or a single job) to collect results for. Alternatively jobs can also be an integer vector of process IDs. If omitted collect will wait for all currently existing children.
wait	if set to FALSE it checks for any results that are available within timeout seconds from now, otherwise it waits for all specified jobs to finish.
timeout	timeout (in seconds) to check for job results - applies only if wait is FALSE.
intermediate	FALSE or a function which will be called while collect waits for results. The function will be called with one parameter which is the list of results received so far.

Details

parallel evaluates the expr expression in parallel to the current R process. Everything is shared read-only (or in fact copy-on-write) between the parallel process and the current process, i.e. no side-effects of the expression affect the main process. The result of the parallel execution can be collected using collect function.

collect function collects any available results from parallel jobs (or in fact any child process). If wait is TRUE then collect waits for all specified jobs to finish before returning a list containing the last reported result for each job. If wait is FALSE then collect merely checks for any results available at the moment and will not wait for jobs to finish. If jobs is specified, jobs not listed there will not be affected or acted upon.

Note: If expr uses low-level multicore functions such as sendMaster a single job can deliver results multiple times and it is the responsibility of the user to interpret them correctly. collect will return NULL for a terminating job that has sent its results already after which the job is no longer available.

Value

parallel returns an object of the class parallelJob which is in turn a childProcess.

collect returns any results that are available in a list. The results will have the same order as the specified jobs. If there are multiple jobs and a job has a name it will be used to name the result, otherwise its process ID will be used.

Author(s)

Simon Urbanek

See Also

mclapply, sendMaster

Examples

  p <- parallel(1:10)
  q <- parallel(1:20)
  collect(list(p, q)) # wait for jobs to finish and collect all results

  p <- parallel(1:10)
  collect(p, wait=FALSE, 10) # will retrieve the result (since it's fast)
  collect(p, wait=FALSE) # will signal the job as terminating
  collect(p, wait=FALSE) # there is no such job

  # a naive parallelized lapply can be created using parallel alone:
  jobs <- lapply(1:10, function(x) parallel(rnorm(x), name=x))
  collect(jobs)

jonclayden/multicore documentation built on May 19, 2019, 7:30 p.m.