jobqueue_class: Assigns Jobs to Workers (R6 Class)
In jobqueue: Run Interruptible Code Asynchronously
jobqueue_class R Documentation
Assigns Jobs to Workers (R6 Class)
Description
Jobs go in. Results come out.
Active bindings
hooksA named list of currently registered callback hooks.
jobsGet or set - List of jobs currently managed by this
jobqueue.
stateThe jobqueue's state: 'starting', 'idle',
'busy', 'stopped', or 'error.'
uidA short string, e.g. 'Q1', that uniquely identifies this
jobqueue.
tmpThe jobqueue's temporary directory.
workersGet or set - List of workers used for processing
jobs.
cndThe error that caused the jobqueue to stop.
Methods
Public methods
-
-
-
-
-
-
-
Method new()
Creates a pool of background processes for handling $run() and
$submit() calls. These workers are initialized
according to the globals, packages, and init arguments.
Usage
jobqueue_class$new(
globals = NULL,
packages = NULL,
namespace = NULL,
init = NULL,
max_cpus = availableCores(),
workers = ceiling(max_cpus * 1.2),
timeout = NULL,
hooks = NULL,
reformat = NULL,
signal = FALSE,
cpus = 1L,
stop_id = NULL,
copy_id = NULL
)
Arguments
globalsA named list of variables that all <job>$exprs will have
access to. Alternatively, an object that can be coerced to a named
list with as.list(), e.g. named vector, data.frame, or environment.
packagesCharacter vector of package names to load on
workers.
namespaceThe name of a package to attach to the
worker's environment.
initA call or R expression wrapped in curly braces to evaluate on
each worker just once, immediately after start-up.
Will have access to variables defined by globals and assets from
packages and namespace. Returned value is ignored.
max_cpusTotal number of CPU cores that can be reserved by all
running jobs (sum(<job>$cpus)). Does not enforce
limits on actual CPU utilization.
workersHow many background worker processes to
start. Set to more than max_cpus to enable standby
workers to quickly swap out with
workers that need to restart.
timeout, hooks, reformat, signal, cpus, stop_id, copy_idDefaults for this jobqueue's $run() method.
Here only, stop_id and copy_id must be either a
function (job) or NULL. hooks can set
jobqueue, worker, and/or
job hooks - see the "Attaching" section in
vignette('hooks').
Returns
A jobqueue object.
Method print()
Print method for a jobqueue.
Usage
jobqueue_class$print(...)
Arguments
...Arguments are not used currently.
Method run()
Creates a job object and submits it to the
jobqueue for running. Any NA arguments will be
replaced with their value from jobqueue_class$new().
Usage
jobqueue_class$run(
expr,
vars = list(),
timeout = NA,
hooks = NA,
reformat = NA,
signal = NA,
cpus = NA,
stop_id = NA,
copy_id = NA,
...
)
Arguments
exprA call or R expression wrapped in curly braces to evaluate on a
worker. Will have access to any variables defined
by vars, as well as the jobqueue's globals,
packages, and init configuration. See vignette('eval').
varsA named list of variables to make available to expr during
evaluation. Alternatively, an object that can be coerced to a named
list with as.list(), e.g. named vector, data.frame, or environment.
Or a function (job) that returns such an object.
timeoutA named numeric vector indicating the maximum number of
seconds allowed for each state the job passes through,
or 'total' to apply a single timeout from 'submitted' to 'done'. Can
also limit the 'starting' state for workers. A
function (job) can be used in place of a number.
Example: timeout = c(total = 2.5, running = 1).
See vignette('stops').
hooksA named list of functions to run when the job
state changes, of the form
hooks = list(created = function (worker) {...}). Or a
function (job) that returns the same. Names of
worker hooks are typically 'created',
'submitted', 'queued', 'dispatched', 'starting', 'running',
'done', or '*' (duplicates okay). See vignette('hooks').
reformatSet reformat = function (job) to define what
<job>$result should return. The default, reformat = NULL passes
<job>$output to <job>$result unchanged.
See vignette('results').
signalShould calling <job>$result signal on condition objects?
When FALSE, <job>$result will return the object without
taking additional action. Setting to TRUE or a character vector of
condition classes, e.g. c('interrupt', 'error', 'warning'), will
cause the equivalent of stop(<condition>) to be called when those
conditions are produced. Alternatively, a function (job) that
returns TRUE or FALSE. See vignette('results').
cpusHow many CPU cores to reserve for this job. Or a
function (job) that returns the same. Used to limit the number of
jobs running simultaneously to respect
<jobqueue>$max_cpus. Does not prevent a job from
using more CPUs than reserved.
stop_idIf an existing job in the
jobqueue has the same stop_id, that
job will be stopped and return an 'interrupt'
condition object as its result. stop_id can also be a
function (job) that returns the stop_id to assign to a given
job. A stop_id of NULL disables this feature.
See vignette('stops').
copy_idIf an existing job in the
jobqueue has the same copy_id, the newly
submitted job will become a "proxy" for that earlier
job, returning whatever result the earlier
job returns. copy_id can also be a function (job)
that returns the copy_id to assign to a given job.
A copy_id of NULL disables this feature.
See vignette('stops').
...Arbitrary named values to add to the returned job
object.
Returns
The new job object.
Method submit()
Adds a job to the jobqueue for
running on a background process.
Usage
jobqueue_class$submit(job)
Arguments
jobA job object, as created by job_class$new().
Returns
This jobqueue, invisibly.
Method wait()
Blocks until the jobqueue enters the given state.
Usage
jobqueue_class$wait(state = "idle", timeout = NULL, signal = TRUE)
Arguments
stateThe name of a jobqueue state. Typically one of:
-
'*' - Every time the state changes.
-
'.next' - Only one time, the next time the state changes.
-
'starting' - workers are starting.
-
'idle' - All workers are ready/idle.
-
'busy' - At least one worker is busy.
-
'stopped' - Shutdown is complete.
timeoutStop the jobqueue if it takes longer
than this number of seconds, or NULL.
signalRaise an error if encountered (will also be recorded in
<jobqueue>$cnd).
Returns
This jobqueue, invisibly.
Method on()
Attach a callback function to execute when the
jobqueue enters state.
Usage
jobqueue_class$on(state, func)
Arguments
stateThe name of a jobqueue state. Typically one of:
-
'*' - Every time the state changes.
-
'.next' - Only one time, the next time the state changes.
-
'starting' - workers are starting.
-
'idle' - All workers are ready/idle.
-
'busy' - At least one worker is busy.
-
'stopped' - Shutdown is complete.
funcA function that accepts a jobqueue object
as input. Return value is ignored.
Returns
A function that when called removes this callback from the
jobqueue.
Method stop()
Stop all jobs and workers.
Usage
jobqueue_class$stop(reason = "jobqueue shut down by user", cls = NULL)
Arguments
reasonPassed to <job>$stop() for any jobs
currently managed by this jobqueue.
clsPassed to <job>$stop() for any jobs currently
managed by this jobqueue.
Returns
This jobqueue, invisibly.
jobqueue documentation built on June 8, 2025, 12:45 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| jobqueue_class | R Documentation |
Assigns Jobs to Workers (R6 Class)
Description
Jobs go in. Results come out.
Active bindings
hooksA named list of currently registered callback hooks.
jobsGet or set - List of jobs currently managed by this
jobqueue.stateThe
jobqueue'sstate:'starting','idle','busy','stopped', or'error.'uidA short string, e.g.
'Q1', that uniquely identifies thisjobqueue.tmpThe
jobqueue's temporary directory.workersGet or set - List of
workersused for processingjobs.cndThe error that caused the
jobqueueto stop.
Methods
Public methods
Method new()
Creates a pool of background processes for handling $run() and
$submit() calls. These workers are initialized
according to the globals, packages, and init arguments.
Usage
jobqueue_class$new( globals = NULL, packages = NULL, namespace = NULL, init = NULL, max_cpus = availableCores(), workers = ceiling(max_cpus * 1.2), timeout = NULL, hooks = NULL, reformat = NULL, signal = FALSE, cpus = 1L, stop_id = NULL, copy_id = NULL )
Arguments
globalsA named list of variables that all
<job>$exprs will have access to. Alternatively, an object that can be coerced to a named list withas.list(), e.g. named vector, data.frame, or environment.packagesCharacter vector of package names to load on
workers.namespaceThe name of a package to attach to the
worker'senvironment.initA call or R expression wrapped in curly braces to evaluate on each
workerjust once, immediately after start-up. Will have access to variables defined byglobalsand assets frompackagesandnamespace. Returned value is ignored.max_cpusTotal number of CPU cores that can be reserved by all running
jobs(sum(<job>$cpus)). Does not enforce limits on actual CPU utilization.workersHow many background
workerprocesses to start. Set to more thanmax_cpusto enable standbyworkersto quickly swap out withworkersthat need to restart.timeout, hooks, reformat, signal, cpus, stop_id, copy_idDefaults for this
jobqueue's$run()method. Here only,stop_idandcopy_idmust be either afunction (job)orNULL.hookscan setjobqueue,worker, and/orjobhooks - see the "Attaching" section invignette('hooks').
Returns
A jobqueue object.
Method print()
Print method for a jobqueue.
Usage
jobqueue_class$print(...)
Arguments
...Arguments are not used currently.
Method run()
Creates a job object and submits it to the
jobqueue for running. Any NA arguments will be
replaced with their value from jobqueue_class$new().
Usage
jobqueue_class$run( expr, vars = list(), timeout = NA, hooks = NA, reformat = NA, signal = NA, cpus = NA, stop_id = NA, copy_id = NA, ... )
Arguments
exprA call or R expression wrapped in curly braces to evaluate on a
worker. Will have access to any variables defined byvars, as well as thejobqueue'sglobals,packages, andinitconfiguration. Seevignette('eval').varsA named list of variables to make available to
exprduring evaluation. Alternatively, an object that can be coerced to a named list withas.list(), e.g. named vector, data.frame, or environment. Or afunction (job)that returns such an object.timeoutA named numeric vector indicating the maximum number of seconds allowed for each state the
jobpasses through, or 'total' to apply a single timeout from 'submitted' to 'done'. Can also limit the 'starting' state forworkers. Afunction (job)can be used in place of a number. Example:timeout = c(total = 2.5, running = 1). Seevignette('stops').hooksA named list of functions to run when the
jobstate changes, of the formhooks = list(created = function (worker) {...}). Or afunction (job)that returns the same. Names ofworkerhooks are typically'created','submitted','queued','dispatched','starting','running','done', or'*'(duplicates okay). Seevignette('hooks').reformatSet
reformat = function (job)to define what<job>$resultshould return. The default,reformat = NULLpasses<job>$outputto<job>$resultunchanged. Seevignette('results').signalShould calling
<job>$resultsignal on condition objects? WhenFALSE,<job>$resultwill return the object without taking additional action. Setting toTRUEor a character vector of condition classes, e.g.c('interrupt', 'error', 'warning'), will cause the equivalent ofstop(<condition>)to be called when those conditions are produced. Alternatively, afunction (job)that returnsTRUEorFALSE. Seevignette('results').cpusHow many CPU cores to reserve for this
job. Or afunction (job)that returns the same. Used to limit the number ofjobsrunning simultaneously to respect<jobqueue>$max_cpus. Does not prevent ajobfrom using more CPUs than reserved.stop_idIf an existing
jobin thejobqueuehas the samestop_id, thatjobwill be stopped and return an 'interrupt' condition object as its result.stop_idcan also be afunction (job)that returns thestop_idto assign to a givenjob. Astop_idofNULLdisables this feature. Seevignette('stops').copy_idIf an existing
jobin thejobqueuehas the samecopy_id, the newly submittedjobwill become a "proxy" for that earlierjob, returning whatever result the earlierjobreturns.copy_idcan also be afunction (job)that returns thecopy_idto assign to a givenjob. Acopy_idofNULLdisables this feature. Seevignette('stops')....Arbitrary named values to add to the returned
jobobject.
Returns
The new job object.
Method submit()
Adds a job to the jobqueue for
running on a background process.
Usage
jobqueue_class$submit(job)
Arguments
jobA
jobobject, as created byjob_class$new().
Returns
This jobqueue, invisibly.
Method wait()
Blocks until the jobqueue enters the given state.
Usage
jobqueue_class$wait(state = "idle", timeout = NULL, signal = TRUE)
Arguments
stateThe name of a
jobqueuestate. Typically one of:-
'*'- Every time the state changes. -
'.next'- Only one time, the next time the state changes. -
'starting'-workersare starting. -
'idle'- Allworkersare ready/idle. -
'busy'- At least oneworkeris busy. -
'stopped'- Shutdown is complete.
-
timeoutStop the
jobqueueif it takes longer than this number of seconds, orNULL.signalRaise an error if encountered (will also be recorded in
<jobqueue>$cnd).
Returns
This jobqueue, invisibly.
Method on()
Attach a callback function to execute when the
jobqueue enters state.
Usage
jobqueue_class$on(state, func)
Arguments
stateThe name of a
jobqueuestate. Typically one of:-
'*'- Every time the state changes. -
'.next'- Only one time, the next time the state changes. -
'starting'-workersare starting. -
'idle'- Allworkersare ready/idle. -
'busy'- At least oneworkeris busy. -
'stopped'- Shutdown is complete.
-
funcA function that accepts a
jobqueueobject as input. Return value is ignored.
Returns
A function that when called removes this callback from the
jobqueue.
Method stop()
Stop all jobs and workers.
Usage
jobqueue_class$stop(reason = "jobqueue shut down by user", cls = NULL)
Arguments
reasonPassed to
<job>$stop()for anyjobscurrently managed by thisjobqueue.clsPassed to
<job>$stop()for anyjobscurrently managed by thisjobqueue.
Returns
This jobqueue, invisibly.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.