JobExperiment: Jobs and Experiments

In mllg/batchtools: Tools for Computation on Batch Systems

Jobs and Experiments

Description

Jobs and Experiments are abstract objects which hold all information necessary to execute a single computational job for a Registry or ExperimentRegistry, respectively.

They can be created using the constructor makeJob which takes a single job id. Jobs and Experiments are passed to reduce functions like reduceResults. Furthermore, Experiments can be used in the functions of the Problem and Algorithm. Jobs and Experiments hold these information:

job.id

Job ID as integer.

pars

Job parameters as named list. For ExperimentRegistry, the parameters are divided into the sublists “prob.pars” and “algo.pars”.

seed

Seed which is set via doJobCollection as scalar integer.

resources

Computational resources which were set for this job as named list.

external.dir

Path to a directory which is created exclusively for this job. You can store external files here. Directory is persistent between multiple restarts of the job and can be cleaned by calling resetJobs.

fun

Job only: User function passed to batchMap.

prob.name

Experiments only: Problem id.

algo.name

Experiments only: Algorithm id.

problem

Experiments only: Problem.

instance

Experiments only: Problem instance.

algorithm

Experiments only: Algorithm.

repl

Experiments only: Replication number.

Note that the slots “pars”, “fun”, “algorithm” and “problem” lazy-load required files from the file system and construct the object on the first access. The realizations are cached for all slots except “instance” (which might be stochastic).

Jobs and Experiments can be executed manually with execJob.

Usage

makeJob(id, reader = NULL, reg = getDefaultRegistry())

Arguments

id	[integer(1) or data.table] Single integer to specify the job or a data.table with column job.id and exactly one row.
reader	[RDSReader | NULL] Reader object to retrieve files. Used internally to cache reading from the file system. The default (NULL) does not make use of caching.
reg	[Registry] Registry. If not explicitly passed, uses the default registry (see setDefaultRegistry).

Value

[Job | Experiment].

Examples

tmp = makeRegistry(file.dir = NA, make.default = FALSE)
batchMap(function(x, y) x + y, x = 1:2, more.args = list(y = 99), reg = tmp)
submitJobs(resources = list(foo = "bar"), reg = tmp)
job = makeJob(1, reg = tmp)
print(job)

# Get the parameters:
job$pars

# Get the job resources:
job$resources

# Execute the job locally:
execJob(job)

mllg/batchtools documentation built on April 21, 2023, 11:49 a.m.