batchtools_custom: Batchtools futures for custom batchtools configuration

In HenrikBengtsson/future.batchtools: A Future API for Parallel and Distributed Processing using 'batchtools'

Batchtools futures for custom batchtools configuration

Description

Batchtools futures for custom batchtools configuration

Usage

batchtools_custom(
  expr,
  envir = parent.frame(),
  substitute = TRUE,
  globals = TRUE,
  label = NULL,
  resources = list(),
  workers = NULL,
  conf.file = findConfFile(),
  cluster.functions = NULL,
  registry = list(),
  ...
)

Arguments

expr	The R expression to be evaluated
envir	The environment in which global environment should be located.
substitute	Controls whether expr should be substitute():d or not.
globals	(optional) a logical, a character vector, a named list, or a Globals object. If TRUE, globals are identified by code inspection based on expr and tweak searching from environment envir. If FALSE, no globals are used. If a character vector, then globals are identified by lookup based their names globals searching from environment envir. If a named list or a Globals object, the globals are used as is.
label	(optional) Label of the future (where applicable, becomes the job name for most job schedulers).
resources	(optional) A named list passed to the batchtools template (available as variable resources). See Section 'Resources' in batchtools::submitJobs() more details.
workers	(optional) The maximum number of workers the batchtools backend may use at any time. Interactive and "local" backends can only process one future at the time (workers = 1L), whereas HPC backends, where futures are resolved via separate jobs on a scheduler, can have multiple workers. In the latter, the default is workers = NULL, which will resolve to getOption("future.batchtools.workers"). If neither are specified, then the default is 100.
conf.file	(character) A batchtools configuration file as for instance returned by batchtools::findConfFile().
cluster.functions	A ClusterFunctions object.
registry	(optional) A named list of settings to control the setup of the batchtools registry.
...	Additional arguments passed to BatchtoolsFuture().

Value

An object of class BatchtoolsFuture.

Examples

options(error = function(...) {
  print(traceback())
})

cf <- batchtools::makeClusterFunctionsInteractive(external = TRUE)
print(cf)
str(cf)
plan(batchtools_custom, cluster.functions = cf)
print(plan())
print(nbrOfWorkers())

## Create explicit future
f <- future({
  cat("PID:", Sys.getpid(), "\n")
  42L
})
print(f)
v <- value(f)
print(v)

options(error = NULL)


## Create explicit future
f <- future({
  cat("PID:", Sys.getpid(), "\n")
  42L
})
print(f)
v <- value(f)
print(v)



## Create explicit future
f <- future({
  cat("PID:", Sys.getpid(), "\n")
  42L
})
v <- value(f)
print(v)

HenrikBengtsson/future.batchtools documentation built on Feb. 1, 2024, 7:47 p.m.