View source: R/tar_make_clustermq.R
tar_make_clustermq | R Documentation |
clustermq
workers.Superseded. Use tar_make()
with crew
:
https://books.ropensci.org/targets/crew.html.
tar_make_clustermq(
names = NULL,
shortcut = targets::tar_config_get("shortcut"),
reporter = targets::tar_config_get("reporter_make"),
seconds_meta_append = targets::tar_config_get("seconds_meta_append"),
seconds_meta_upload = targets::tar_config_get("seconds_meta_upload"),
seconds_reporter = targets::tar_config_get("seconds_reporter"),
seconds_interval = targets::tar_config_get("seconds_interval"),
workers = targets::tar_config_get("workers"),
log_worker = FALSE,
callr_function = callr::r,
callr_arguments = targets::tar_callr_args_default(callr_function, reporter),
envir = parent.frame(),
script = targets::tar_config_get("script"),
store = targets::tar_config_get("store"),
garbage_collection = targets::tar_config_get("garbage_collection")
)
names |
Names of the targets to run or check. Set to |
shortcut |
Logical of length 1, how to interpret the |
reporter |
Character of length 1, name of the reporter to user.
Controls how messages are printed as targets run in the pipeline.
Defaults to
|
seconds_meta_append |
Positive numeric of length 1 with the minimum
number of seconds between saves to the local metadata and progress files
in the data store.
Higher values generally make the pipeline run faster, but unsaved
work (in the event of a crash) is not up to date.
When the pipeline ends,
all the metadata and progress data is saved immediately,
regardless of |
seconds_meta_upload |
Positive numeric of length 1 with the minimum
number of seconds between uploads of the metadata and progress data
to the cloud
(see https://books.ropensci.org/targets/cloud-storage.html).
Higher values generally make the pipeline run faster, but unsaved
work (in the event of a crash) may not be backed up to the cloud.
When the pipeline ends,
all the metadata and progress data is uploaded immediately,
regardless of |
seconds_reporter |
Positive numeric of length 1 with the minimum number of seconds between times when the reporter prints progress messages to the R console. |
seconds_interval |
Deprecated on 2023-08-24 (version 1.2.2.9001).
Use |
workers |
Positive integer, number of persistent |
log_worker |
Logical, whether to write a log file for each worker.
Same as the |
callr_function |
A function from |
callr_arguments |
A list of arguments to |
envir |
An environment, where to run the target R script
(default: The |
script |
Character of length 1, path to the
target script file. Defaults to |
store |
Character of length 1, path to the
|
garbage_collection |
Logical of length 1, whether to run garbage
collection on the main process before sending a target to a worker.
Independent from the |
tar_make_clustermq()
is like tar_make()
except that targets
run in parallel on persistent workers. A persistent worker is an
R process that runs for a long time and runs multiple
targets during its lifecycle. Persistent
workers launch as soon as the pipeline reaches an outdated
target with deployment = "worker"
, and they keep running
until the pipeline starts to wind down.
To configure tar_make_clustermq()
, you must configure
the clustermq
package. To do this, set global options
clustermq.scheduler
and clustermq.template
inside the target script file (default: _targets.R
).
To read more about configuring clustermq
for your scheduler, visit
https://mschubert.github.io/clustermq/articles/userguide.html#configuration # nolint
or https://books.ropensci.org/targets/hpc.html.
clustermq
is not a strict dependency of targets
,
so you must install clustermq
yourself.
NULL
except if callr_function = callr::r_bg()
, in which case
a handle to the callr
background process is returned. Either way,
the value is invisibly returned.
Several functions like tar_make()
, tar_read()
, tar_load()
,
tar_meta()
, and tar_progress()
read or modify
the local data store of the pipeline.
The local data store is in flux while a pipeline is running,
and depending on how distributed computing or cloud computing is set up,
not all targets can even reach it. So please do not call these
functions from inside a target as part of a running
pipeline. The only exception is literate programming
target factories in the tarchetypes
package such as tar_render()
and tar_quarto()
.
Other pipeline:
tar_make()
,
tar_make_future()
if (!identical(tolower(Sys.info()[["sysname"]]), "windows")) {
if (identical(Sys.getenv("TAR_EXAMPLES"), "true")) { # for CRAN
tar_dir({ # tar_dir() runs code from a temp dir for CRAN.
tar_script({
library(targets)
library(tarchetypes)
options(clustermq.scheduler = "multiprocess") # Does not work on Windows.
tar_option_set()
list(tar_target(x, 1 + 1))
}, ask = FALSE)
tar_make_clustermq()
})
}
}
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.