makeRegistry | R Documentation |
makeRegistry
constructs the inter-communication object for all functions in batchtools
.
All communication transactions are processed via the file system:
All information required to run a job is stored as JobCollection
in a file in the
a subdirectory of the file.dir
directory.
Each jobs stores its results as well as computational status information (start time, end time, error message, ...)
also on the file system which is regular merged parsed by the master using syncRegistry
.
After integrating the new information into the Registry, the Registry is serialized to the file system via saveRegistry
.
Both syncRegistry
and saveRegistry
are called whenever required internally.
Therefore it should be safe to quit the R session at any time.
Work can later be resumed by calling loadRegistry
which de-serializes the registry from
the file system.
The registry created last is saved in the package namespace (unless make.default
is set to
FALSE
) and can be retrieved via getDefaultRegistry
.
Canceled jobs and jobs submitted multiple times may leave stray files behind.
These can be swept using sweepRegistry
.
clearRegistry
completely erases all jobs from a registry, including log files and results,
and thus allows you to start over.
makeRegistry(
file.dir = "registry",
work.dir = getwd(),
conf.file = findConfFile(),
packages = character(0L),
namespaces = character(0L),
source = character(0L),
load = character(0L),
seed = NULL,
make.default = TRUE
)
file.dir |
[ If you pass |
work.dir |
[ The provided path will get normalized unless it is given relative to the home directory (i.e., starting with “~”). Note that some templates do not handle relative paths well. |
conf.file |
[ The file lookup is implemented in the internal (but exported) function
Set to |
packages |
[ |
namespaces |
[ |
source |
[ |
load |
[ |
seed |
[ |
make.default |
[ |
Currently batchtools understands the following options set via the configuration file:
cluster.functions
:As returned by a constructor, e.g. makeClusterFunctionsSlurm
.
default.resources
:List of resources to use. Will be overruled by resources specified via submitJobs
.
temp.dir
:Path to directory to use for temporary registries.
sleep
:Custom sleep function. See waitForJobs
.
expire.after
:Number of iterations before treating jobs as expired in waitForJobs
.
compress
:Compression algorithm to use via saveRDS
.
[environment
] of class “Registry” with the following slots:
file.dir
[path]:File directory.
work.dir
[path]:Working directory.
temp.dir
[path]:Temporary directory. Used if file.dir
is NA
to create temporary registries.
packages
[character()]:Packages to load on the slaves.
namespaces
[character()]:Namespaces to load on the slaves.
seed
[integer(1)]:Registry seed. Before each job is executed, the seed seed + job.id
is set.
cluster.functions
[cluster.functions]:Usually set in your conf.file
. Set via a call to makeClusterFunctions
. See example.
default.resources
[named list()]:Usually set in your conf.file
. Named list of default resources.
max.concurrent.jobs
[integer(1)]:Usually set in your conf.file
. Maximum number of concurrent jobs for a single user and current registry on the system.
submitJobs
will try to respect this setting. The resource “max.concurrent.jobs” has higher precedence.
defs
[data.table]:Table with job definitions (i.e. parameters).
status
[data.table]:Table holding information about the computational status. Also see getJobStatus
.
resources
[data.table]:Table holding information about the computational resources used for the job. Also see getJobResources
.
tags
[data.table]:Table holding information about tags. See Tags.
hash
[character(1)]:Unique hash which changes each time the registry gets saved to the file system. Can be utilized to invalidate the cache of knitr.
Other Registry:
clearRegistry()
,
getDefaultRegistry()
,
loadRegistry()
,
removeRegistry()
,
saveRegistry()
,
sweepRegistry()
,
syncRegistry()
tmp = makeRegistry(file.dir = NA, make.default = FALSE)
print(tmp)
# Set cluster functions to interactive mode and start jobs in external R sessions
tmp$cluster.functions = makeClusterFunctionsInteractive(external = TRUE)
# Change packages to load
tmp$packages = c("MASS")
saveRegistry(reg = tmp)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.