BatchtoolsFutureBackend: A batchtools future is a future whose value will be resolved...

In future.batchtools: A Future API for Parallel and Distributed Processing using 'batchtools'

A batchtools future is a future whose value will be resolved via batchtools

Description

A batchtools local future is a synchronous uniprocess future that will be evaluated in a background R session. A batchtools interactive future is a synchronous uniprocess future that will be evaluated in the current R session (and variables will be assigned to the calling environment rather than to a local one). Both types of futures will block until the futures are resolved.

A batchtools SSH future is an asynchronous multiprocess future that will be evaluated in a background R session.

We highly recommend using future::multisession (sic!) futures of the future package instead of SSH batchtools futures.

Usage

BatchtoolsFutureBackend(
  workers = 1L,
  resources = list(),
  finalize = getOption("future.finalize", TRUE),
  cluster.functions = NULL,
  registry = list(),
  conf.file = findConfFile(),
  interrupts = TRUE,
  delete = getOption("future.batchtools.delete", "on-success"),
  ...
)

BatchtoolsCustomFutureBackend(...)

BatchtoolsInteractiveFutureBackend(fs.latency = 0, ...)

BatchtoolsLocalFutureBackend(fs.latency = 0, ...)

BatchtoolsSSHFutureBackend(workers = availableWorkers(), fs.latency = 65, ...)

Arguments

workers	(optional) The maximum number of workers the batchtools backend may use at any time. Interactive and "local" backends can only process one future at the time (workers = 1), whereas HPC backends, where futures are resolved via separate jobs on a scheduler, can have multiple workers. In the latter, the default is workers = NULL, which will resolve to getOption("future.batchtools.workers", 100).
resources	(optional) A named list passed to the batchtools job-script template as variable resources. This is based on how batchtools::submitJobs() works, with the exception for specially reserved names defined by the future.batchtools package; resources[["details"]], if TRUE, results in the job script outputting job details and job summaries at the beginning and at the end. resources[["startup"]] and resources[["shutdown"]] are character vectors of shell code to be injected to the job script as-is. resources[["modules"]] is character vector of Linux environment modules to be loaded. resources[["envs"]], is an optional names character vector specifying environment variables to be set. resources[["rscript"]] is an optional character vector specifying how the 'Rscript' is launched. The resources[["rscript_args"]] field is an optional character vector specifying the 'Rscript' command-line arguments. resources[["asis"]] is a character vector that are passed as-is to the job script and are injected as job resource declarations. All remaining resources named elements are injected as named resource specification for the scheduler.
finalize	If TRUE, a future's batchtools Registry is automatically deleted when the future is garbage collected, otherwise not.
cluster.functions	(optional) Assigned as-is to the each future's batchtools Registry.
registry	(optional) A named list of settings applied to each future's batchtools Registry. This is a more convenient alternative to using argument conf.file.
conf.file	(optional) A "batchtools-configuration" R script, which is sourced when each future's batchtools Registry is created. Any variables created by this script is assigned to the registry. The default file is the one found by batchtools::findConfFile(), if any.
interrupts	If FALSE, attempts to interrupt futures will not take place on this backend, even if the backend supports it. This is useful when, for instance, it takes a long time to interrupt a future.
delete	Controls if and when the batchtools job registry folder is deleted. If "on-success" (default), it is deleted if the future was resolved successfully and the expression did not produce an error. If "never", then it is never deleted. If "always", then it is always deleted.
fs.latency	[numeric(1)] Expected maximum latency of the file system, in seconds. Set to a positive number for network file systems like NFS which enables more robust (but also more expensive) mechanisms to access files and directories. Usually safe to set to 0 to disable the heuristic, e.g. if you are working on a local file system.
...	Additional arguments passed to BatchtoolsFutureBackend().

Details

Batchtools local futures rely on the batchtools backend set up by batchtools::makeClusterFunctionsInteractive(external = TRUE) and batchtools interactive futures on the one set up by batchtools::makeClusterFunctionsInteractive(). These are supported by all operating systems.

An alternative to batchtools local futures is to use cluster futures of the future package with a single local background session, i.e. plan(cluster, workers = "localhost").

An alternative to batchtools interactive futures is to use plan(sequential, split = TRUE) futures of the future package.

Batchtools SSH futures rely on the batchtools backend set up by batchtools::makeClusterFunctionsSSH(). The batchtools SSH backend only works on operating systems supporting the ssh and ps command-line tool, e.g. Linux and macOS.

Value

A future::FutureBackend object of class BatchtoolsFutureBackend

An object of class BatchtoolsFuture.

An object of class BatchtoolsUniprocessFuture.

Examples

library(future)

## Create custom cluster functions (here same as "local")
cf <- batchtools::makeClusterFunctionsInteractive(external = TRUE)
print(cf)
str(cf)

# Use custom batchtools backend
plan(future.batchtools::batchtools_custom, cluster.functions = cf)
print(plan())

message("Main process ID: ", Sys.getpid())

f <- future({
  data.frame(
    hostname = Sys.info()[["nodename"]],
          os = Sys.info()[["sysname"]],
       cores = unname(parallelly::availableCores()),
         pid = Sys.getpid(),
     modules = Sys.getenv("LOADEDMODULES")
  )
})
info <- value(f)
print(info)

future.batchtools documentation built on Oct. 19, 2026, 1:07 a.m.