JobBatch: Class representing a batch job

Description Usage Arguments Details Fields and Methods Author(s) Examples

Description

Package: R.batch
Class JobBatch

Object
~~|
~~+--JobBatch

Directly known subclasses:

public static class JobBatch
extends Object

Class representing a batch job. A JobBatch has one or several Job:s.

To run batch jobs, most often this class the only one needed. The Job class is only to investigate details about a specific job.

Usage

1
JobBatch(root="jobs", ...)

Arguments

root

A name of a job root directory.

...

Not used.

Details

When a job is processed then following happens in order:

  1. A non-locked job from the "todo/" directory will be retrieved.

  2. The job will be moved to the "running/" directory.

  3. The job will be locked (a lock file is created and opened).

  4. If any of the above fails, NULL is returned.

  5. The job is initiated; source code in the "src/" directory followed by the job directory is loaded. Here onRun() must be defined. All other onNNN() functions maybe be redefined, otherwise default ones are used. If there is syntax error in the source code, the job is moved to the "erroneous/" directory.

  6. The working directory is set to the directory of the job.

  7. If a stored image (typically from a previously interrupted job) is detected, it is loaded into the current job and onRestart() is called.

  8. The job is started and onStart() is called.

  9. onRun() is called.

  10. If sucessful, the job is moved to "finished/" and is unlocked (the lock file is removed).

  11. The Job object that was processed is returned.

In addition, for step 7-9: If an error occurs, onError() followed by onFinally() are called and the job is moved to the "failed/" directory. If an interrupt occurs, onInterrupt() followed by onFinally() are called and the job is moved to the "interrupted/" directory. By default, onInterrupt() save an image of the job, by calling saveImage(job). In any case the job will be unlock and returned.

Note that, if the job directory is "locked" by another process, which can happen if someone browser the job directory or similar, it cannot be moved. If this happends when a job is moved to another directory, the move operation will be tried 10 times every 10 seconds. If the job was not moved an error is generated (and the job remains in its current directory).

Fields and Methods

Methods:

as.character Gets a character string representation of the job batch.
checkRequirements Checks that requirements are fulfilled or not.
clean Cleans up among jobs in this JobBatch.
copyFrom Copies a job batch directory into this one.
createStub Creates a jobs directory structure stub.
findJobs Searches by name for Job:s in this JobBatch.
getDirectory Gets a subdirectory of the job batch.
getErroneousPath -
getFailedPath -
getFinishedPath -
getInputPath -
getInterruptedPath -
getNextJob Gets next non-locked job.
getOutputPath -
getRoot Gets the root path of the job batch.
getRunAndFinishJob Gets an non-locked job, process it and moves it to a final destination.
getRunningPath -
getSrcPath -
getSummary Gets a summary of the jobs directory.
getTodoPath -
main Static method to be called from the command line.
moveJobTo Moves the job to another directory.
print Prints a summary of the jobs directory.
resetJobs Resets Jobs in the job batch.
run Process some or all jobs available.
setRoot -
setupDemo Static method to setup a demo job batch directory structure.
validate Validates the job batch.

Methods inherited from Object:
$, $<-, [[, [[<-, as.character, attach, attachLocally, clearCache, clearLookupCache, clone, detach, equals, extend, finalize, gc, getEnvironment, getFieldModifier, getFieldModifiers, getFields, getInstantiationTime, getStaticInstance, hasField, hashCode, ll, load, objectSize, print, registerFinalizer, save

Author(s)

Henrik Bengtsson (http://www.braju.com/R/)

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# To prevent R from asking us to press ENTER before each image
options(device.ask.default=FALSE)

# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Setup a demo job directory
# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
rootPath <- JobBatch$setupDemo("mandelbrot", overwrite=TRUE)
batch <- JobBatch(rootPath)

# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Now, imaging that this code was running on several
# different host all with access to the job directory.
# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
print(batch)

# Process jobs until no more exist or no more
# can be retrieved.
run(batch, verbose=-150)

print(batch)

print(list.files(getOutputPath(batch)))


# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# ONLY FOR REDUNDANCY TESTING OF THE PACKAGE
# - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Assert that jobs in batch ended up where expected.
cat("Validation JobBatch results...\n");
expected <- list(erroneous="job05", failed="job04", interrupted=NULL,
                    finished=c("job01", "job02", "job03"), running=NULL);
for (name in names(expected)) {
  jobs <- getSummary(batch)[[name]]$jobs;
  if (!identical(jobs, expected[[name]])) {
    msg <- paste("Final directory '", name,
                 "' does not contain expected jobs: ",
                 paste(jobs, collapse=", "), " != ",
                 paste(expected[[name]], collapse=","));
    cat(msg, "\n");
    stop(msg);
  }
}
cat("Validation JobBatch results...done\n");

R.batch documentation built on May 2, 2019, 4:58 p.m.