plan: Plan how to resolve a future

In HenrikBengtsson/future: Unified Parallel and Distributed Processing in R for Everyone

Plan how to resolve a future

Description

This function allows the user to plan the future, more specifically, it specifies how future():s are resolved, e.g. sequentially or in parallel.

Usage

plan(
  strategy = NULL,
  ...,
  substitute = TRUE,
  .skip = FALSE,
  .call = TRUE,
  .cleanup = TRUE,
  .init = TRUE
)

Arguments

strategy	The evaluation function (or name of it) to use for resolving a future. If NULL, then the current strategy is returned.
...	Additional arguments overriding the default arguments of the evaluation function. Which additional arguments are supported depends on what evaluation function is used, e.g. several support argument workers but not all. For details, see the individual functions of which some are linked to below.
substitute	If TRUE, the strategy expression is substitute():d, otherwise not.
.skip	(internal) If TRUE, then attempts to set a strategy that is the same as what is currently in use, will be skipped.
.call	(internal) Used for recording the call to this function.
.cleanup	(internal) Used to stop implicitly started clusters.
.init	(internal) Used to initiate workers.

Details

The default strategy is sequential, but the default can be configured by option future.plan and, if that is not set, system environment variable R_FUTURE_PLAN. To reset the strategy back to the default, use plan("default").

Value

If a new strategy is chosen, then the previous one is returned (invisible), otherwise the current one is returned (visibly).

Built-in evaluation strategies

The future package provides the following built-in backends:

sequential:

Resolves futures sequentially in the current R process, e.g. plan(sequential).

multisession:

Resolves futures asynchronously (in parallel) in separate R sessions running in the background on the same machine, e.g. plan(multisession) and plan(multisession, workers = 2).

multicore:

Resolves futures asynchronously (in parallel) in separate forked R processes running in the background on the same machine, e.g. plan(multicore) and plan(multicore, workers = 2). This backend is not supported on Windows.

cluster:

Resolves futures asynchronously (in parallel) in separate R sessions running typically on one or more machines, e.g. plan(cluster), plan(cluster, workers = 2), and plan(cluster, workers = c("n1", "n1", "n2", "server.remote.org")).

Other package provide additional evaluation strategies. For example, the future.callr package implements an alternative to the multisession backend on top of the callr package, e.g. plan(future.callr::callr, workers = 2). Another example is the future.batchtools package, which implements, on top of the batchtools package, e.g. plan(future.batchtools::batchtools_slurm). These types of futures are resolved via job schedulers, which typically are available on high-performance compute (HPC) clusters, e.g. LSF, Slurm, TORQUE/PBS, Sun Grid Engine, and OpenLava.

To "close" any background workers (e.g. multisession), change the plan to something different; plan(sequential) is recommended for this.

For package developers

Please refrain from modifying the future strategy inside your packages / functions, i.e. do not call plan() in your code. Instead, leave the control on what backend to use to the end user. This idea is part of the core philosophy of the future framework—as a developer you can never know what future backends the user have access to. Moreover, by not making any assumptions about what backends are available, your code will also work automatically with any new backends developed after you wrote your code.

If you think it is necessary to modify the future strategy within a function, then make sure to undo the changes when exiting the function. This can be done using:

  oplan <- plan(new_set_of_strategies)
  on.exit(plan(oplan), add = TRUE)
  [...]

This is important because the end-user might have already set the future strategy elsewhere for other purposes and will most likely not known that calling your function will break their setup. Remember, your package and its functions might be used in a greater context where multiple packages and functions are involved and those might also rely on the future framework, so it is important to avoid stepping on others' toes.

Using plan() in scripts and vignettes

When writing scripts or vignettes that use futures, try to place any call to plan() as far up (i.e. as early on) in the code as possible. This will help users to quickly identify where the future plan is set up and allow them to modify it to their computational resources. Even better is to leave it to the user to set the plan() prior to source():ing the script or running the vignette. If a ‘.future.R’ exists in the current directory and / or in the user's home directory, it is sourced when the future package is loaded. Because of this, the ‘.future.R’ file provides a convenient place for users to set the plan(). This behavior can be controlled via an R option—see future options for more details.

Examples

a <- b <- c <- NA_real_

# An sequential future
plan(sequential)
f <- future({
  a <- 7
  b <- 3
  c <- 2
  a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs


# A sequential future with lazy evaluation
plan(sequential)
f <- future({
  a <- 7
  b <- 3
  c <- 2
  a * b * c
}, lazy = TRUE)
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs


# A multicore future (specified as a string)
plan("multicore")
f <- future({
  a <- 7
  b <- 3
  c <- 2
  a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs

## Multisession futures gives an error on R CMD check on
## Windows (but not Linux or macOS) for unknown reasons.
## The same code works in package tests.


# A multisession future (specified via a string variable)
plan("future::multisession")
f <- future({
  a <- 7
  b <- 3
  c <- 2
  a * b * c
})
y <- value(f)
print(y)
str(list(a = a, b = b, c = c)) ## All NAs




## Explicitly specifying number of workers
## (default is parallelly::availableCores())
plan(multicore, workers = 2)
message("Number of parallel workers: ", nbrOfWorkers())


## Explicitly close multisession workers by switching plan
plan(sequential)

HenrikBengtsson/future documentation built on Dec. 15, 2024, 8:42 p.m.