batchtools: Tools for Computation on Batch Systems

As a successor of the packages 'BatchJobs' and 'BatchExperiments', this package provides a parallel implementation of the Map function for high performance computing systems managed by schedulers 'IBM Spectrum LSF' (<http://www-03.ibm.com/systems/spectrum-computing/products/lsf/>), 'OpenLava' (<http://www.openlava.org/>), 'Univia Grid Engine'/'Oracle Grid Engine' (<http://www.univa.com/>), 'Slurm' (<http://slurm.schedmd.com/>), 'Torque/PBS' (<http://www.adaptivecomputing.com/products/open-source/torque/>), or 'Docker Swarm' (<https://docs.docker.com/swarm/>). A multicore and socket mode allow the parallelization on a local machines, and multiple machines can be hooked up via SSH to create a makeshift cluster. Moreover, the package provides an abstraction mechanism to define large-scale computer experiments in a well-organized and reproducible way.

Author
Michel Lang [cre, aut], Bernd Bischl [aut], Dirk Surmann [ctb]
Date of publication
2016-11-08 00:59:52
Maintainer
Michel Lang <michellang@gmail.com>
License
LGPL-3
Version
0.9.0
URLs

View on CRAN

Man pages

addExperiments
Add Experiments to the Registry
batchExport
Export Objects to the Slaves
batchMap
Map Operation for Batch Systems
batchMapResults
Map Over Results to Create New Jobs
batchtools-package
Tools for Computation on Batch Systems
btlapply
Synchronous Apply Functions
cfBrewTemplate
Cluster Functions Helper to Write Job Description Files
cfHandleUnknownSubmitError
Cluster Functions Helper to Handle Unknown Errors
cfKillJob
Cluster Functions Helper to Kill Batch Jobs
cfReadBrewTemplate
Cluster Functions Helper to Parse a Brew Template
chunkIds
Chunk Jobs for Sequential Execution
doJobCollection
Execute Jobs of a JobCollection
execJob
Execute a Single Jobs
ExperimentRegistry
ExperimentRegistry Constructor
findJobs
Find and Filter Jobs
getErrorMessages
Retrieve Error Messages
getJobTable
Query Job Information
getStatus
Summarize the Computational Status
grepLogs
Grep Log Files for a Pattern
JobCollection
JobCollection Constructor
JobExperiment
Jobs and Experiments
JoinTables
Inner, Left, Right, Outer, Semi and Anti Join for Data Tables
killJobs
Kill Jobs
loadResult
Load the Result of a Single Job
makeClusterFunctions
ClusterFunctions Constructor
makeClusterFunctionsDocker
ClusterFunctions for Docker
makeClusterFunctionsInteractive
ClusterFunctions for Sequential Execution in the Running R...
makeClusterFunctionsLSF
ClusterFunctions for LSF Systems
makeClusterFunctionsMulticore
ClusterFunctions for Parallel Multicore Execution
makeClusterFunctionsOpenLava
ClusterFunctions for OpenLava
makeClusterFunctionsSGE
ClusterFunctions for SGE Systems
makeClusterFunctionsSlurm
ClusterFunctions for Slurm Systems
makeClusterFunctionsSocket
ClusterFunctions for Parallel Socket Execution
makeClusterFunctionsSSH
ClusterFunctions for Remote SSH Execution
makeClusterFunctionsTorque
ClusterFunctions for OpenPBS/Torque Systems
makeSubmitJobResult
Create a SubmitJobResult
ProblemAlgorithm
Define Problems and Algorithms for Experiments
reduceResults
Reduce Results
reduceResultsList
Apply Functions on Results
Registry
Registry Constructor
removeExperiments
Remove Experiments
resetJobs
Reset the Computational State of Jobs
runHook
Trigger Evaluation of Custom Function
runOSCommand
Run OS Commands on Local or Remote Machines
showLog
Inspect Log Files
submitJobs
Submit Jobs to the Batch Systems
summarizeExperiments
Quick Summary over Experiments
Tags
Add or Remove Job Tags
testJob
Run Jobs Interactively
waitForJobs
Wait for Termination of Jobs
Worker
Create a Linux-Worker

Files in this package

batchtools
batchtools/inst
batchtools/inst/CITATION
batchtools/inst/bin
batchtools/inst/bin/linux-helper
batchtools/inst/templates
batchtools/inst/templates/torque_lido.tmpl
batchtools/inst/templates/slurm_simple.tmpl
batchtools/inst/templates/lsf_simple.tmpl
batchtools/inst/templates/openlava_simple.tmpl
batchtools/inst/templates/sge_simple.tmpl
batchtools/inst/templates/slurm_dortmund.tmpl
batchtools/inst/templates/testJob.tmpl
batchtools/inst/doc
batchtools/inst/doc/ExamplePiSim.Rmd
batchtools/inst/doc/Setup.R
batchtools/inst/doc/ExampleExperiment.html
batchtools/inst/doc/ErrorHandling.html
batchtools/inst/doc/ExampleExperiment.Rmd
batchtools/inst/doc/Setup.html
batchtools/inst/doc/Setup.Rmd
batchtools/inst/doc/ExamplePiSim.R
batchtools/inst/doc/ErrorHandling.Rmd
batchtools/inst/doc/ErrorHandling.R
batchtools/inst/doc/ExamplePiSim.html
batchtools/inst/doc/ExampleExperiment.R
batchtools/tests
batchtools/tests/testthat.R
batchtools/tests/testthat
batchtools/tests/testthat/test_manual.R
batchtools/tests/testthat/test_seed.R
batchtools/tests/testthat/test_ClusterFunctionsSSH.R
batchtools/tests/testthat/test_export.R
batchtools/tests/testthat/test_doJobCollection.R
batchtools/tests/testthat/test_JobCollection.R
batchtools/tests/testthat/test_summarizeExperiments.R
batchtools/tests/testthat/test_runOSCommand.R
batchtools/tests/testthat/test_testJob.R
batchtools/tests/testthat/test_Results.R
batchtools/tests/testthat/test_ClusterFunctionsSocket.R
batchtools/tests/testthat/test_ExperimentRegistry.R
batchtools/tests/testthat/test_submitJobs.R
batchtools/tests/testthat/test_resetJobs.R
batchtools/tests/testthat/test_addProblem.R
batchtools/tests/testthat/test_waitForJobs.R
batchtools/tests/testthat/test_addExperiments.R
batchtools/tests/testthat/test_btlapply.R
batchtools/tests/testthat/test_Registry.R
batchtools/tests/testthat/test_getJobTable.R
batchtools/tests/testthat/test_chunkIds.R
batchtools/tests/testthat/test_count.R
batchtools/tests/testthat/test_killJobs.R
batchtools/tests/testthat/test_findConfFile.R
batchtools/tests/testthat/test_addAlgorithm.R
batchtools/tests/testthat/test_ClusterFunctionsMulticore.R
batchtools/tests/testthat/test_getStatus.R
batchtools/tests/testthat/test_batchMap.R
batchtools/tests/testthat/test_findJobs.R
batchtools/tests/testthat/test_getErrorMessages.R
batchtools/tests/testthat/test_Job.R
batchtools/tests/testthat/test_removeExperiments.R
batchtools/tests/testthat/helper.R
batchtools/tests/testthat/test_convertIds.R
batchtools/tests/testthat/test_memory.R
batchtools/tests/testthat/test_ClusterFunctions.R
batchtools/tests/testthat/test_hooks.R
batchtools/tests/testthat/test_tags.R
batchtools/tests/testthat/test_parallelMap.R
batchtools/tests/testthat/test_joins.R
batchtools/tests/testthat/test_showLog.R
batchtools/tests/testthat/test_grepLogs.R
batchtools/src
batchtools/src/count_not_missing.c
batchtools/src/fill_gaps.c
batchtools/NAMESPACE
batchtools/NEWS.md
batchtools/R
batchtools/R/clusterFunctionsSlurm.R
batchtools/R/clusterFunctionsInteractive.R
batchtools/R/clusterFunctions.R
batchtools/R/btlapply.R
batchtools/R/Registry.R
batchtools/R/reduceResults.R
batchtools/R/killJobs.R
batchtools/R/Logs.R
batchtools/R/helpers.R
batchtools/R/JobTables.R
batchtools/R/clusterFunctionsOpenLava.R
batchtools/R/Worker.R
batchtools/R/Job.R
batchtools/R/ProblemAlgorithm.R
batchtools/R/getStatus.R
batchtools/R/getErrorMessages.R
batchtools/R/chunkIds.R
batchtools/R/execJob.R
batchtools/R/Tags.R
batchtools/R/Joins.R
batchtools/R/submitJobs.R
batchtools/R/clusterFunctionsMulticore.R
batchtools/R/ExperimentRegistry.R
batchtools/R/Export.R
batchtools/R/clusterFunctionsSocket.R
batchtools/R/JobCollection.R
batchtools/R/updateRegisty.R
batchtools/R/clusterFunctionsSGE.R
batchtools/R/loadResult.R
batchtools/R/testJob.R
batchtools/R/waitForJobs.R
batchtools/R/clusterFunctionsDocker.R
batchtools/R/clusterFunctionsSSH.R
batchtools/R/summarizeExperiments.R
batchtools/R/clusterFunctionsLSF.R
batchtools/R/addExperiments.R
batchtools/R/removeExperiments.R
batchtools/R/clusterFunctionsTorque.R
batchtools/R/resetJobs.R
batchtools/R/batchMapResults.R
batchtools/R/findJobs.R
batchtools/R/doJobCollection.R
batchtools/R/ids.R
batchtools/R/batchMap.R
batchtools/R/zzz.R
batchtools/R/Hooks.R
batchtools/vignettes
batchtools/vignettes/ExamplePiSim.Rmd
batchtools/vignettes/ExampleExperiment.Rmd
batchtools/vignettes/Setup.Rmd
batchtools/vignettes/ErrorHandling.Rmd
batchtools/README.md
batchtools/MD5
batchtools/build
batchtools/build/vignette.rds
batchtools/DESCRIPTION
batchtools/man
batchtools/man/reduceResultsList.Rd
batchtools/man/makeClusterFunctionsLSF.Rd
batchtools/man/getJobTable.Rd
batchtools/man/ProblemAlgorithm.Rd
batchtools/man/Tags.Rd
batchtools/man/Registry.Rd
batchtools/man/makeClusterFunctionsMulticore.Rd
batchtools/man/killJobs.Rd
batchtools/man/getErrorMessages.Rd
batchtools/man/makeClusterFunctionsSGE.Rd
batchtools/man/batchExport.Rd
batchtools/man/btlapply.Rd
batchtools/man/batchMapResults.Rd
batchtools/man/cfHandleUnknownSubmitError.Rd
batchtools/man/addExperiments.Rd
batchtools/man/loadResult.Rd
batchtools/man/batchMap.Rd
batchtools/man/runHook.Rd
batchtools/man/waitForJobs.Rd
batchtools/man/execJob.Rd
batchtools/man/makeSubmitJobResult.Rd
batchtools/man/batchtools-package.Rd
batchtools/man/Worker.Rd
batchtools/man/summarizeExperiments.Rd
batchtools/man/reduceResults.Rd
batchtools/man/JobCollection.Rd
batchtools/man/getStatus.Rd
batchtools/man/grepLogs.Rd
batchtools/man/makeClusterFunctions.Rd
batchtools/man/submitJobs.Rd
batchtools/man/JobExperiment.Rd
batchtools/man/makeClusterFunctionsInteractive.Rd
batchtools/man/makeClusterFunctionsSocket.Rd
batchtools/man/removeExperiments.Rd
batchtools/man/makeClusterFunctionsSlurm.Rd
batchtools/man/makeClusterFunctionsDocker.Rd
batchtools/man/findJobs.Rd
batchtools/man/testJob.Rd
batchtools/man/makeClusterFunctionsOpenLava.Rd
batchtools/man/showLog.Rd
batchtools/man/chunkIds.Rd
batchtools/man/cfKillJob.Rd
batchtools/man/doJobCollection.Rd
batchtools/man/makeClusterFunctionsTorque.Rd
batchtools/man/runOSCommand.Rd
batchtools/man/ExperimentRegistry.Rd
batchtools/man/makeClusterFunctionsSSH.Rd
batchtools/man/resetJobs.Rd
batchtools/man/cfBrewTemplate.Rd
batchtools/man/JoinTables.Rd
batchtools/man/cfReadBrewTemplate.Rd