Job-class: Class representing a set of jobs, each which is the...
In mskilab/Flow: Workflow and task management for genomics pipelines.

Job-class

R Documentation

Class representing a set of jobs, each which is the combination of a Task and entity (i.e. a sample, pair, or set)

Description

S4 class for Job

Class Job defines a set of nrow(entities) firehose jobs "myjobs" which is an instantiation of an Task combined with one or more rows of an (keyed) annotation data.table entities. An Job is created from an Task + data.table of annotations, and (optional) output directory (otherwise made default). Once created, several things can be done with an Job: It's local command line tasks can be extracted with cmd(myjobs), it's bsub tasks can be extracted with bcmd(myjobs). These can be dumped to a file or executed directory from R via a system command on the cmd(), bcmd(), qcmd() output, or run with run(myjobs), brun(myjobs), qrun(myjobs)...

Once jobs are running / completing the status of jobs can be checked with update(.Object). In this case the timestamp of each output will be updated and the relevant annotations will be updated after querying the output directories. The data.table associated with the Job can be accessed using the runinfo command i.e. runinfo(.Object)

Replacing a subset of Job object with other compatibler Job items. "compatible" job items will be from the identical task and will contain entities already contained in the Job object.

getting output data.table associated with Job object

As jobs complete, one may want to update a "master" data.table with the outputs of Jobs. This can be useful for manual running of larger workflows to which a given task contributes.

#@exportMethod merge

Usage

update(object, ...)

## S4 replacement method for signature 'Job'
x[i, id = FALSE] <- value

module(.Object)

task(.Object)

mem(.Object)

time(.Object)

io_c(.Object)

io_n(.Object)

qprior(.Object)

nice_val(.Object)

queue(.Object)

qos(.Object)

cores(.Object)

gres(.Object)

now(.Object)

## S4 method for signature 'Job'
runinfo(.Object)

report(.Object, ...)

merge(x, y, ...)

Arguments

`x`	data.table or Job
`y`	data.table or Job
`force`	logical flag whether to force overwrite
`prefix`	prefix to add to columns merged from the Job
`suffix`	suffix to add to columns merged from the Job
`sep`	separator to add to columns merged from the Job

Details

Replacing a subset of Job object with another (compatible Job object)

Merge Job output annotations with data.table

Slots

task: Task object that this job was built from
rootdir: scalar character directory specifying root output directory of jobs
runinfo: keyed data.table with length(.Object) rows containing columns $outdir = output directory for each job $queue = character specifying the queue on which this job should be run $mem = numeric specifying G of max memory to run this job with $cmd = command line to execute job locally $bcmd = command line to submit job to farm $status = character that is either "completed", "ready", "not ready", or "outdated" if time stamp of one or more input annotation >= timestamp of the output annotation. $status.info = description of status when not ready, which can describe missing inputs or updated inputs since last run
inputs: keyed data.table with length(.Object) rows with column corresponding to key and columns specifying values input annotations for each entity
stamps: keyed data.table with length(.Object) rows with column corresponding to key and columns specifying timestamps of input annotations for each entity
outputs: keyed data.table with length(.Object) rows $timestamp column for time stamp of output (or NA if job is not complete) and columns for table key + 0 or more output annotations with filled in values if job is complete or NA