batchtools: Tools for Computation on Batch Systems
Version 0.9.3

As a successor of the packages 'BatchJobs' and 'BatchExperiments', this package provides a parallel implementation of the Map function for high performance computing systems managed by schedulers 'IBM Spectrum LSF' (), 'OpenLava' (), 'Univa Grid Engine'/'Oracle Grid Engine' (), 'Slurm' (), 'TORQUE/PBS' (), or 'Docker Swarm' (). A multicore and socket mode allow the parallelization on a local machines, and multiple machines can be hooked up via SSH to create a makeshift cluster. Moreover, the package provides an abstraction mechanism to define large-scale computer experiments in a well-organized and reproducible way.

Browse man pages Browse package API and functions Browse package files

AuthorMichel Lang [cre, aut], Bernd Bischl [aut], Dirk Surmann [ctb]
Date of publication2017-04-21 22:50:18 UTC
MaintainerMichel Lang <michellang@gmail.com>
LicenseLGPL-3
Version0.9.3
URL https://github.com/mllg/batchtools
Package repositoryView on CRAN
InstallationInstall the latest version of this package by entering the following in R:
install.packages("batchtools")

Man pages

addAlgorithm: Define Algorithms for Experiments
addExperiments: Add Experiments to the Registry
addProblem: Define Problems for Experiments
batchExport: Export Objects to the Slaves
batchMap: Map Operation for Batch Systems
batchMapResults: Map Over Results to Create New Jobs
batchReduce: Reduce Operation for Batch Systems
batchtools-deprecated: Deprecated function in the batchtools package
batchtools-package: batchtools: Tools for Computation on Batch Systems
btlapply: Synchronous Apply Functions
cfBrewTemplate: Cluster Functions Helper to Write Job Description Files
cfHandleUnknownSubmitError: Cluster Functions Helper to Handle Unknown Errors
cfKillJob: Cluster Functions Helper to Kill Batch Jobs
cfReadBrewTemplate: Cluster Functions Helper to Parse a Brew Template
chunk: Chunk Jobs for Sequential Execution
chunkIds: Chunk Jobs for Sequential Execution
clearRegistry: Remove All Jobs
doJobCollection: Execute Jobs of a JobCollection
estimateRuntimes: Estimate Remaining Runtimes
execJob: Execute a Single Jobs
findJobs: Find and Filter Jobs
getDefaultRegistry: Get and Set the Default Registry
getErrorMessages: Retrieve Error Messages
getJobTable: Query Job Information
getStatus: Summarize the Computational Status
grepLogs: Grep Log Files for a Pattern
JobCollection: JobCollection Constructor
JobExperiment: Jobs and Experiments
JoinTables: Inner, Left, Right, Outer, Semi and Anti Join for Data Tables
killJobs: Kill Jobs
loadRegistry: Load a Registry from the File System
loadResult: Load the Result of a Single Job
makeClusterFunctions: ClusterFunctions Constructor
makeClusterFunctionsDocker: ClusterFunctions for Docker
makeClusterFunctionsInteractive: ClusterFunctions for Sequential Execution in the Running R...
makeClusterFunctionsLSF: ClusterFunctions for LSF Systems
makeClusterFunctionsMulticore: ClusterFunctions for Parallel Multicore Execution
makeClusterFunctionsOpenLava: ClusterFunctions for OpenLava
makeClusterFunctionsSGE: ClusterFunctions for SGE Systems
makeClusterFunctionsSlurm: ClusterFunctions for Slurm Systems
makeClusterFunctionsSocket: ClusterFunctions for Parallel Socket Execution
makeClusterFunctionsSSH: ClusterFunctions for Remote SSH Execution
makeClusterFunctionsTORQUE: ClusterFunctions for OpenPBS/TORQUE Systems
makeExperimentRegistry: ExperimentRegistry Constructor
makeRegistry: Registry Constructor
makeSubmitJobResult: Create a SubmitJobResult
reduceResults: Reduce Results
reduceResultsList: Apply Functions on Results
removeExperiments: Remove Experiments
removeRegistry: Remove a Registry from the File System
resetJobs: Reset the Computational State of Jobs
runHook: Trigger Evaluation of Custom Function
runOSCommand: Run OS Commands on Local or Remote Machines
saveRegistry: Store the Registy to the File System
showLog: Inspect Log Files
submitJobs: Submit Jobs to the Batch Systems
summarizeExperiments: Quick Summary over Experiments
sweepRegistry: Check Consistency and Remove Obsolete Information
syncRegistry: Synchronize the Registry
Tags: Add or Remove Job Tags
testJob: Run Jobs Interactively
waitForJobs: Wait for Termination of Jobs
Worker: Create a Linux-Worker

Functions

Algorithm Man page
ClusterFunctions Man page
Experiment Man page
ExperimentRegistry Man page
Hook Man page
Hooks Man page
Job Man page
JobCollection Man page
JoinTables Man page
Problem Man page
Registry Man page
Rscript Source code
SubmitJobResult Man page
Tags Man page
Worker Man page
addAlgorithm Man page Source code
addExperiments Man page Source code
addJobTags Man page Source code
addProblem Man page Source code
addlevel Source code
ajoin Man page Source code
allIds Source code
assertExperimentRegistry Source code
assertRegistry Source code
auto_increment Source code
batchExport Man page Source code
batchMap Man page Source code
batchMapResults Man page Source code
batchMapResultsWrapper Source code
batchReduce Man page Source code
batchReduceWrapper Source code
batchtools Man page
batchtools-deprecated Man page
batchtools-package Man page
binpack Man page Source code
btlapply Man page Source code
btmapply Man page Source code
castIds Source code
catf Source code
cfBrewTemplate Man page Source code
cfHandleUnknownSubmitError Man page Source code
cfKillJob Man page Source code
cfReadBrewTemplate Man page Source code
chintersect Source code
chsetdiff Source code
chunk Man page Source code
chunkIds Man page Source code
clearRegistry Man page Source code
convertId Source code
convertIds Source code
count Source code
createCollection Source code
default.sleep Source code
doJobCollection Man page Source code
doJobCollection.JobCollection Source code
doJobCollection.character Source code
estimateRuntimes Man page Source code
execJob Man page Source code
execJob.Experiment Source code
execJob.Job Source code
execJob.JobCollection Source code
execJob.character Source code
extractLog Source code
filter Source code
filterNull Source code
findConfFile Source code
findDone Man page Source code Source code
findErrors Man page Source code Source code
findExperiments Man page Source code
findExpired Man page Source code Source code
findJobs Man page Source code
findNotDone Man page Source code Source code
findNotStarted Man page Source code Source code
findNotSubmitted Man page Source code Source code
findOnSystem Man page Source code Source code
findQueued Man page Source code
findRunning Man page Source code
findStarted Man page Source code Source code
findSubmitted Man page Source code Source code
findTagged Man page Source code
findTemplateFile Source code
getAlgorithmIds Man page Source code
getAlgorithmURI Source code
getBatchIds Source code
getDefaultRegistry Man page Source code
getErrorMessages Man page Source code
getExternalDirs Source code
getExternalPath Source code
getJob Source code
getJob.ExperimentCollection Source code
getJob.JobCollection Source code
getJobFiles Source code
getJobPars Man page Source code
getJobPars.ExperimentRegistry Source code
getJobPars.Registry Source code
getJobPath Source code
getJobResources Man page Source code
getJobStatus Man page Source code
getJobTable Man page Source code
getJobTags Man page Source code
getLog Man page Source code
getLogFiles Source code
getLogPath Source code
getProblemIds Man page Source code
getProblemURI Source code
getResultFiles Source code
getResultPath Source code
getSeed Source code
getSleepFunction Source code
getStatus Man page Source code
getStatusTable Source code
getUpdatePath Source code
getUsedJobTags Man page Source code
grepLogs Man page Source code
guessBy Source code
ijoin Man page Source code
info Source code
insert Source code
is.error Source code
killJobs Man page Source code
ljoin Man page Source code
loadRegistry Man page Source code
loadRegistryDependencies Source code
loadResult Man page Source code
lpt Man page Source code
makeClusterFunctions Man page Source code
makeClusterFunctionsDocker Man page Source code
makeClusterFunctionsInteractive Man page Source code
makeClusterFunctionsLSF Man page Source code
makeClusterFunctionsMulticore Man page Source code
makeClusterFunctionsOpenLava Man page Source code
makeClusterFunctionsSGE Man page Source code
makeClusterFunctionsSSH Man page Source code
makeClusterFunctionsSlurm Man page Source code
makeClusterFunctionsSocket Man page Source code
makeClusterFunctionsTORQUE Man page Source code
makeExperimentRegistry Man page Source code
makeJob Man page Source code
makeJob.ExperimentRegistry Source code
makeJob.Registry Source code
makeJobCollection Man page Source code
makeJobCollection.ExperimentRegistry Source code
makeJobCollection.Registry Source code
makeProgressBar Source code
makeRegistry Man page Source code
makeSubmitJobResult Man page Source code
mangle Source code
mergeRegistries Source code
mergedJobs Source code
names2 Source code
noIds Source code
npath Source code
ojoin Man page Source code
onLoad Source code
onUnload Source code
print.ClusterFunctions Source code
print.ExperimentRegistry Source code
print.JobCollection Source code
print.Registry Source code
print.RuntimeEstimate Man page Source code
print.Status Source code
print.SubmitJobResult Source code
readLog Source code
reduceResults Man page Source code
reduceResultsDataTable Man page Source code
reduceResultsList Man page Source code Source code
removeAlgorithms Man page Source code
removeExperiments Man page Source code
removeJobTags Man page Source code
removeProblems Man page Source code
removeRegistry Man page Source code
resetJobs Man page Source code
rjoin Man page Source code
rmlevel Source code
runHook Man page Source code
runHook.JobCollection Source code
runHook.Registry Source code
runOSCommand Man page Source code
saveRegistry Man page Source code
seq_row Source code
setClasses Source code
setDefaultRegistry Man page Source code
setKey Source code
setSystemConf Source code
showLog Man page Source code
sjoin Man page Source code
stopf Source code
stri_trunc Source code
submitJobs Man page Source code
summarizeExperiments Man page Source code
sweepRegistry Man page Source code
syncRegistry Man page Source code
testJob Man page Source code
ujoin Man page Source code
unmangle Source code
updateRegistry Source code
ustamp Source code
vcapply Source code
viapply Source code
vlapply Source code
vnapply Source code
waitForFiles Source code
waitForJobs Man page Source code
waitForResults Source code
warningf Source code
with_seed Source code
writeRDS Source code

Files

inst
inst/CITATION
inst/bin
inst/bin/linux-helper
inst/templates
inst/templates/torque-lido.tmpl
inst/templates/openlava-simple.tmpl
inst/templates/lsf-simple.tmpl
inst/templates/sge-simple.tmpl
inst/templates/slurm-dortmund.tmpl
inst/templates/slurm-simple.tmpl
inst/templates/testJob.tmpl
inst/doc
inst/doc/v20_ErrorHandling.Rmd
inst/doc/v01_Migration.R
inst/doc/v11_ExampleExperiment.html
inst/doc/v00_Setup.Rmd
inst/doc/v11_ExampleExperiment.Rmd
inst/doc/v11_ExampleExperiment.R
inst/doc/v20_ErrorHandling.html
inst/doc/v10_ExamplePiSim.Rmd
inst/doc/v01_Migration.Rmd
inst/doc/v00_Setup.html
inst/doc/v20_ErrorHandling.R
inst/doc/v01_Migration.html
inst/doc/v10_ExamplePiSim.html
inst/doc/v10_ExamplePiSim.R
inst/doc/v00_Setup.R
tests
tests/testthat.R
tests/testthat
tests/testthat/test_manual.R
tests/testthat/test_sweepRegistry.R
tests/testthat/test_seed.R
tests/testthat/test_ClusterFunctionsSSH.R
tests/testthat/test_export.R
tests/testthat/test_estimateRuntimes.R
tests/testthat/test_doJobCollection.R
tests/testthat/test_JobCollection.R
tests/testthat/test_summarizeExperiments.R
tests/testthat/test_runOSCommand.R
tests/testthat/test_testJob.R
tests/testthat/test_Results.R
tests/testthat/test_ClusterFunctionsSocket.R
tests/testthat/test_ExperimentRegistry.R
tests/testthat/test_submitJobs.R
tests/testthat/test_resetJobs.R
tests/testthat/test_addProblem.R
tests/testthat/test_waitForJobs.R
tests/testthat/test_addExperiments.R
tests/testthat/test_btlapply.R
tests/testthat/test_Registry.R
tests/testthat/test_getJobTable.R
tests/testthat/test_chunkIds.R
tests/testthat/test_count.R
tests/testthat/test_killJobs.R
tests/testthat/test_mergeRegistries.R
tests/testthat/test_findConfFile.R
tests/testthat/test_addAlgorithm.R
tests/testthat/test_ClusterFunctionsMulticore.R
tests/testthat/test_getStatus.R
tests/testthat/test_batchMap.R
tests/testthat/test_findJobs.R
tests/testthat/test_getErrorMessages.R
tests/testthat/test_batchReduce.R
tests/testthat/test_chunk.R
tests/testthat/test_Job.R
tests/testthat/test_removeExperiments.R
tests/testthat/helper.R
tests/testthat/test_removeRegistry.R
tests/testthat/test_convertIds.R
tests/testthat/test_memory.R
tests/testthat/test_ClusterFunctions.R
tests/testthat/test_hooks.R
tests/testthat/test_tags.R
tests/testthat/test_parallelMap.R
tests/testthat/test_joins.R
tests/testthat/test_showLog.R
tests/testthat/test_grepLogs.R
src
src/count_not_missing.c
src/fill_gaps.c
src/lpt.c
src/init.c
src/binpack.c
NAMESPACE
NEWS.md
R
R/clusterFunctionsSlurm.R
R/clusterFunctionsInteractive.R
R/clusterFunctions.R
R/btlapply.R
R/Registry.R
R/reduceResults.R
R/killJobs.R
R/sleep.R
R/Logs.R
R/helpers.R
R/JobTables.R
R/clusterFunctionsOpenLava.R
R/Worker.R
R/Job.R
R/Algorithm.R
R/getStatus.R
R/estimateRuntimes.R
R/saveRegistry.R
R/filenames.R
R/Problem.R
R/getErrorMessages.R
R/chunkIds.R
R/execJob.R
R/Tags.R
R/Joins.R
R/runOSCommand.R
R/submitJobs.R
R/clusterFunctionsMulticore.R
R/syncRegistry.R
R/ExperimentRegistry.R
R/Export.R
R/clearRegistry.R
R/clusterFunctionsSocket.R
R/getDefaultRegistry.R
R/clusterFunctionsTORQUE.R
R/JobCollection.R
R/config.R
R/updateRegisty.R
R/clusterFunctionsSGE.R
R/loadResult.R
R/testJob.R
R/waitForJobs.R
R/loadRegistry.R
R/clusterFunctionsDocker.R
R/clusterFunctionsSSH.R
R/batchReduce.R
R/summarizeExperiments.R
R/clusterFunctionsLSF.R
R/addExperiments.R
R/removeExperiments.R
R/removeRegistry.R
R/waitForFiles.R
R/resetJobs.R
R/sweepRegistry.R
R/batchMapResults.R
R/findJobs.R
R/doJobCollection.R
R/ids.R
R/batchMap.R
R/zzz.R
R/Hooks.R
R/mergeRegistries.R
vignettes
vignettes/v20_ErrorHandling.Rmd
vignettes/v00_Setup.Rmd
vignettes/v11_ExampleExperiment.Rmd
vignettes/tikz_prob_algo_simple.png
vignettes/v10_ExamplePiSim.Rmd
vignettes/v01_Migration.Rmd
vignettes/tikz_prob_algo_simple.tex
README.md
MD5
build
build/vignette.rds
DESCRIPTION
man
man/reduceResultsList.Rd
man/makeClusterFunctionsLSF.Rd
man/getJobTable.Rd
man/getDefaultRegistry.Rd
man/Tags.Rd
man/addAlgorithm.Rd
man/makeClusterFunctionsMulticore.Rd
man/killJobs.Rd
man/makeRegistry.Rd
man/getErrorMessages.Rd
man/batchtools-deprecated.Rd
man/makeClusterFunctionsSGE.Rd
man/clearRegistry.Rd
man/batchExport.Rd
man/btlapply.Rd
man/batchMapResults.Rd
man/cfHandleUnknownSubmitError.Rd
man/addExperiments.Rd
man/loadResult.Rd
man/batchMap.Rd
man/runHook.Rd
man/waitForJobs.Rd
man/execJob.Rd
man/makeSubmitJobResult.Rd
man/batchtools-package.Rd
man/Worker.Rd
man/saveRegistry.Rd
man/summarizeExperiments.Rd
man/loadRegistry.Rd
man/reduceResults.Rd
man/JobCollection.Rd
man/getStatus.Rd
man/addProblem.Rd
man/makeExperimentRegistry.Rd
man/grepLogs.Rd
man/makeClusterFunctions.Rd
man/submitJobs.Rd
man/JobExperiment.Rd
man/makeClusterFunctionsInteractive.Rd
man/estimateRuntimes.Rd
man/makeClusterFunctionsSocket.Rd
man/removeExperiments.Rd
man/makeClusterFunctionsSlurm.Rd
man/makeClusterFunctionsDocker.Rd
man/chunk.Rd
man/findJobs.Rd
man/testJob.Rd
man/syncRegistry.Rd
man/makeClusterFunctionsOpenLava.Rd
man/showLog.Rd
man/chunkIds.Rd
man/makeClusterFunctionsTORQUE.Rd
man/cfKillJob.Rd
man/doJobCollection.Rd
man/batchReduce.Rd
man/runOSCommand.Rd
man/sweepRegistry.Rd
man/makeClusterFunctionsSSH.Rd
man/resetJobs.Rd
man/cfBrewTemplate.Rd
man/JoinTables.Rd
man/cfReadBrewTemplate.Rd
man/removeRegistry.Rd
batchtools documentation built on May 19, 2017, 9:59 a.m.