BatchJobs: Batch Computing with R
Version 1.6

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF, SLURM and Sun Grid Engine. Multicore and SSH systems are also supported. For further details see the project web page.

AuthorBernd Bischl <bernd_bischl@gmx.net>, Michel Lang <michellang@gmail.com>, Henrik Bengtsson <henrikb@braju.com>
Date of publication2015-03-18 18:01:21
MaintainerBernd Bischl <bernd_bischl@gmx.net>
LicenseBSD_2_clause + file LICENSE
Version1.6
URL https://github.com/tudo-r/BatchJobs
Package repositoryView on CRAN
InstallationInstall the latest version of this package by entering the following in R:
install.packages("BatchJobs")

Popular man pages

batchExpandGrid: Map function over all combinations.
batchMapQuick: Combination of makeRegistry, batchMap and submitJobs.
batchReduceResults: Reduces results via a binary function and adds jobs for this...
cfReadBrewTemplate: Cluster functions helper: Read in your brew template file.
loadResult: Loads a specific result file.
makeClusterFunctionsMulticore: Use multiple cores on local Linux machine to spawn parallel...
reduceResults: Reduce results from result directory.
See all...

All man pages Function index File listing

Man pages

addRegistryPackages: Add packages to registry.
addRegistrySourceDirs: Add source dirs to registry.
addRegistrySourceFiles: Add source files to registry.
applyJobFunction: applyJobFunction ONLY FOR INTERNAL USAGE.
batchExpandGrid: Map function over all combinations.
batchExport: Export R object to be available on the slaves.
BatchJobs: The BatchJobs package
batchMap: Maps a function over lists or vectors, adding jobs to a...
batchMapQuick: Combination of makeRegistry, batchMap and submitJobs.
batchMapResults: Maps a function over the results of a registry by using...
batchReduce: Reduces via a binary function over a list adding jobs to a...
batchReduceResults: Reduces results via a binary function and adds jobs for this...
batchUnexport: Unload exported R objects.
callFunctionOnSSHWorkers: Call an arbitrary function on specified SSH workers.
cfBrewTemplate: Cluster functions helper: Brew your template into a job...
cfHandleUnknownSubmitError: Cluster functions helper: Handle an unknown error during job...
cfKillBatchJob: Cluster functions helper: Kill a batch job via OS command
cfReadBrewTemplate: Cluster functions helper: Read in your brew template file.
configuration: BatchJobs configuration.
copyRequiredJobFiles: ONLY FOR INTERNAL USAGE.
dbCreateJobDefTable: ONLY FOR INTERNAL USAGE.
dbGetJobs: ONLY FOR INTERNAL USAGE.
debugMulticore: Helper function to debug multicore mode.
debugSSH: Helper function to debug SSH mode.
filterResults: Find all results where a specific condition is true.
findJobs: Finds ids of jobs that match a query.
findState: Find jobs depending on computional state.
getConfig: Returns a list of BatchJobs configuration settings
getErrorMessages: Get error messages of jobs.
getJob: Get job from registry by id.
getJobIds: Get ids of jobs in registry.
getJobInfo: Get computational information of jobs.
getJobNr: Get number of jobs in registry.
getJobParamDf: Returns parameters for all jobs as the rows of a data.frame.
getJobResources: Function to get the resources that were submitted for some...
getJobs: Get jobs from registry by id.
getLogFiles: Get log file paths for jobs.
getResources: Function to get job resources in job function.
getSSHWorkersInfo: Print and return R installation and other information for SSH...
grepLogs: Grep log files for a pattern.
installPackagesOnSSHWorkers: Install packages on SSH workers.
killJobs: Kill some jobs on the batch system.
loadConfig: Load a specific configuration file.
loadExports: Load exported R data objects.
loadRegistry: Load a previously saved registry.
loadResult: Loads a specific result file.
loadResults: Loads result files for id vector.
makeClusterFunctions: Create a ClusterFuntions object.
makeClusterFunctionsInteractive: Create cluster functions for sequential execution in same...
makeClusterFunctionsLocal: Create cluster functions for sequential execution on local...
makeClusterFunctionsLSF: Create cluster functions for LSF systems.
makeClusterFunctionsMulticore: Use multiple cores on local Linux machine to spawn parallel...
makeClusterFunctionsSGE: Create cluster functions for Sun Grid Engine systems.
makeClusterFunctionsSLURM: Create cluster functions for SLURM-based systems.
makeClusterFunctionsSSH: Create an SSH cluster to execute jobs.
makeClusterFunctionsTorque: Create cluster functions for torque-based systems.
makeJob: Creates a job description.
makeRegistry: Construct a registry object.
makeSSHWorker: Create SSH worker for SSH cluster functions.
makeSubmitJobResult: Create a SubmitJobResult object.
reduceResults: Reduce results from result directory.
removeRegistry: Remove a registry object.
removeRegistryPackages: Remove packages from registry.
removeRegistrySourceDirs: Remove packages from registry.
removeRegistrySourceFiles: Remove source files from registry.
resetJobs: Reset computational state of jobs.
sanitizePath: Sanitize a path
setConfig: Set and overwrite configuration settings
setJobFunction: Sets the job function for already existing jobs.
setJobNames: Set job names.
showClusterStatus: Show information about available computational resources on...
showLog: Display the contents of a log file.
showStatus: Retrieve or show status information about jobs.
sourceRegistryFiles: Source registry files
submitJobs: Submit jobs or chunks of jobs to batch system via cluster...
sweepRegistry: Sweep obsolete files from the file system.
testJob: Tests a job by running it with Rscript in a new process.
updateRegistry: ONLY FOR INTERNAL USAGE.
waitForJobs: Wait for termination of jobs on the batch system.

Functions

.BatchJobs.R Man page
BatchJobs Man page
BatchJobs-package Man page
ClusterFunctions Man page
Job Man page
Registry Man page
SSHWorker Man page
SubmitJobResult Man page
addIntModulo Source code
addRegistryPackages Man page Source code
addRegistrySourceDirs Man page Source code
addRegistrySourceFiles Man page Source code
adjustRegistryPaths Source code
applyJobFunction Man page Source code
applyJobFunction.Registry Source code
assignConf Source code
assignConfDefaults Source code
batchApply Source code
batchApplyWrapper Source code
batchExpandGrid Man page Source code
batchExport Man page Source code
batchMap Man page Source code
batchMapQuick Man page Source code
batchMapResults Man page Source code
batchMapResultsWrapper Source code
batchReduce Man page Source code
batchReduceResults Man page Source code
batchReduceResultsWrapper Source code
batchReduceWrapper Source code
batchUnexport Man page Source code
buffer Source code
callFunctionOnSSHWorkers Man page Source code
cfBrewTemplate Man page Source code
cfHandleUnknownSubmitError Man page Source code
cfKillBatchJob Man page Source code
cfReadBrewTemplate Man page Source code
checkConf Source code
checkConfElements Source code
checkDir Source code
checkDirs Source code
checkId Source code
checkIdValid Source code
checkIds Source code
checkMoreArgs Source code
checkPart Source code
checkRegistry Source code
checkUserFunction Source code
configuration Man page
convertUseNames Source code
copyRequiredJobFiles Man page Source code
copyRequiredJobFiles.Registry Source code
createShardedDirs Source code
dbAddData Source code
dbCheckJobIds Source code
dbConnectToJobsDB Source code
dbConvertNumericToPOSIXct Source code
dbCreateExpandedJobsView Source code
dbCreateJobDefTable Man page Source code
dbCreateJobDefTable.Registry Source code
dbCreateJobStatusTable Source code
dbDoQueries Source code
dbDoQuery Source code
dbFindDisappeared Source code
dbFindDone Source code
dbFindErrors Source code
dbFindExpiredJobs Source code
dbFindOnSystem Source code
dbFindRunning Source code
dbFindStarted Source code
dbFindSubmitted Source code
dbFindSubmittedNotTerminated Source code
dbFindTerminated Source code
dbGetConnection Source code
dbGetConnection.SQLiteDriver Source code
dbGetErrorMsgs Source code
dbGetExpandedJobsTable Source code
dbGetFirstJobInChunkIds Source code
dbGetJobCount Source code
dbGetJobId Source code
dbGetJobIds Source code
dbGetJobIdsIfAllDone Source code
dbGetJobNames Source code
dbGetJobStatusTable Source code
dbGetJobs Man page Source code
dbGetJobs.Registry Source code
dbGetLastAddedIds Source code
dbGetStats Source code
dbMakeMessageDone Source code
dbMakeMessageError Source code
dbMakeMessageKilled Source code
dbMakeMessageStarted Source code
dbMakeMessageSubmitted Source code
dbMatchJobNames Source code
dbRemoveJobs Source code
dbSelectWithIds Source code
dbSendMessage Source code
dbSendMessages Source code
dbSetJobFunction Source code
dbSetJobNames Source code
debugMulticore Man page Source code
debugSSH Man page Source code
doJob Source code
fileExists Source code
filterResults Man page Source code
findConfigs Source code
findDisappeared Man page Source code
findDone Man page Source code
findErrors Man page Source code
findExpired Man page Source code
findHelperScriptLinux Source code
findJobs Man page Source code
findMissingResults Man page Source code
findNotDone Man page Source code
findNotErrors Man page Source code
findNotOnSystem Man page Source code
findNotRunning Man page Source code
findNotStarted Man page Source code
findNotSubmitted Man page Source code
findNotTerminated Man page Source code
findOnSystem Man page Source code
findRunning Man page Source code
findStarted Man page Source code
findState Source code
findSubmitted Man page Source code
findTerminated Man page Source code
findWorker Source code
getArgNames Source code
getBatchIds Source code
getBatchJobsConf Source code
getClusterFunctions Source code
getConfFilePath Source code
getConfNames Source code
getConfig Man page Source code
getErrorMessages Man page Source code
getExportDir Source code
getFilePaths Source code
getFunDir Source code
getFunFilePath Source code
getJob Man page Source code
getJobDirs Source code
getJobIds Man page Source code
getJobInfo Man page Source code
getJobInfo.Registry Source code
getJobInfoInternal Source code
getJobNr Man page Source code
getJobParamDf Man page Source code
getJobParentDir Source code
getJobResources Man page Source code
getJobs Man page Source code
getJobs.Registry Source code
getKillJob Source code
getListJobs Source code
getLogFilePath Source code
getLogFiles Man page Source code
getPendingDir Source code
getPendingFile Source code
getProgressBar Source code
getRScriptFilePath Source code
getRScripts Source code
getRandomSeed Source code
getRegistryFilePath Source code
getResources Man page Source code
getResourcesDir Source code
getResourcesFilePath Source code
getResult Source code
getResultFilePath Source code
getResults Source code
getSSHWorkersInfo Man page Source code
getStatus Man page Source code
getWorkerNumberOfCPUs Source code
getWorkerNumberOfCPUs.WorkerLinux Source code
getWorkerSchedulerStatus Source code
getWorkerStatus Source code
getWorkerStatus.WorkerLinux Source code
grepLogs Man page Source code
info Source code
installPackagesOnSSHWorkers Man page Source code
is.accessible Source code
isOnSlave Source code
isPathFromRoot Source code
isRegistryDir Source code
killJobs Man page Source code
killWorkerJob Source code
killWorkerJob.WorkerLinux Source code
listWorkerJobs Source code
listWorkerJobs.WorkerLinux Source code
loadConf Source code
loadConfig Man page Source code
loadExports Man page Source code
loadRegistry Man page Source code
loadResult Man page Source code
loadResults Man page Source code
makeClusterFunctions Man page Source code
makeClusterFunctionsInteractive Man page Source code
makeClusterFunctionsLSF Man page Source code
makeClusterFunctionsLocal Man page Source code
makeClusterFunctionsMulticore Man page Source code
makeClusterFunctionsSGE Man page Source code
makeClusterFunctionsSLURM Man page Source code
makeClusterFunctionsSSH Man page Source code
makeClusterFunctionsTorque Man page Source code
makeJob Man page Source code
makeRegistry Man page Source code
makeRegistryInternal Source code
makeSSHWorker Man page Source code
makeSubmitJobResult Man page Source code
makeWorker Source code
makeWorkerLocalLinux Source code
makeWorkerRemoteLinux Source code
now Source code
onAttach Source code
onLoad Source code
print.ClusterFunctions Source code
print.Config Source code
print.Job Source code
print.Registry Source code
print.SubmitJobResult Source code
printableConf Source code
readConfs Source code
readSQLFile Source code
reduceResults Man page Source code
reduceResultsDataFrame Man page Source code
reduceResultsList Man page Source code
reduceResultsMatrix Man page Source code
reduceResultsVector Man page Source code
removeDirs Source code
removeRegistry Man page Source code
removeRegistryPackages Man page Source code
removeRegistrySourceDirs Man page Source code
removeRegistrySourceFiles Man page Source code
resetJobs Man page Source code
resrc Source code
runOSCommandLinux Source code
runWorkerCommand Source code
sanitizePath Man page Source code
saveConf Source code
saveFunction Source code
saveRegistry Source code
saveResources Source code
seeder Source code
sendMail Source code
setConfig Man page Source code
setJobFunction Man page Source code
setJobFunction.Registry Source code
setJobNames Man page Source code
setOnSlave Source code
showClusterStatus Man page Source code
showLog Man page Source code
showStatus Man page Source code
sourceConfFile Source code
sourceConfFiles Source code
sourceRegistryFiles Man page Source code
sourceRegistryFilesInternal Source code
sqlQuote Source code
startWorkerJob Source code
startWorkerJob.WorkerLinux Source code
submitJobs Man page Source code
sweepRegistry Man page Source code
switchWd Source code
syncRegistry Source code
testJob Man page Source code
updateRegistry Man page Source code
updateRegistry.Registry Source code
updateWorker Source code
useStagedQueries Source code
waitForFiles Source code
waitForJobs Man page Source code
writeRscripts Source code
writeSQLFile Source code

Files

inst
inst/CITATION
inst/bin
inst/bin/linux-helper
inst/etc
inst/etc/BatchJobs_global_config.R
tests
tests/run-all.R
tests/testthat
tests/testthat/test_sweepRegistry.R
tests/testthat/test_export.R
tests/testthat/helpers.R
tests/testthat/test_reduceResults.R
tests/testthat/test_getJobInfo.R
tests/testthat/test_sanitizePath.R
tests/testthat/test_testJob.R
tests/testthat/test_filterResults.R
tests/testthat/test_getJobParamDf.R
tests/testthat/test_batchMapQuick.R
tests/testthat/test_loadResults.R
tests/testthat/test_submitJobs.R
tests/testthat/test_resources.R
tests/testthat/test_resetJobs.R
tests/testthat/test_getLogFiles.R
tests/testthat/test_waitForJobs.R
tests/testthat/test_batchMap.R
tests/testthat/test_setJobFunction.R
tests/testthat/test_findJobs.R
tests/testthat/test_showStatus.R
tests/testthat/test_batchReduce.R
tests/testthat/test_findStatus.R
tests/testthat/test_batchMapResults.R
tests/testthat/test_batchReduceResults.R
tests/testthat/test_sourceRegistryFiles.R
tests/testthat/test_zzz_cleanup.R
tests/testthat/test_getJob.R
tests/testthat/test_database.R
tests/testthat/test_sqlquotes.R
tests/testthat/test_batchExpandGrid.R
tests/testthat/test_packages.R
tests/testthat/test_makeRegistry.R
tests/testthat/test_grepLogs.R
tests/testthat/test_doJob.R
NAMESPACE
NEWS
R
R/getJobInfo.R
R/buffer.R
R/clusterFunctionsInteractive.R
R/clusterFunctions.R
R/Registry.R
R/filterResults.R
R/reduceResults.R
R/killJobs.R
R/database.R
R/writeRscripts.R
R/callFunctionOnSSHWorkers.R
R/getJobParamDf.R
R/helpers.R
R/Worker.R
R/Job.R
R/conf.R
R/debugMulticore.R
R/clusterFunctionsHelpers.R
R/showLog.R
R/debugSSH.R
R/batchApply.R
R/batchExpandGrid.R
R/filenames.R
R/Packages.R
R/getErrorMessages.R
R/loadResults.R
R/setJobNames.R
R/sanitizePath.R
R/findStatus.R
R/submitJobs.R
R/batchReduceResults.R
R/clusterFunctionsMulticore.R
R/Exports.R
R/syncRegistry.R
R/getLogFiles.R
R/sourceRegistryFiles.R
R/showStatus.R
R/sendMail.R
R/clusterFunctionsSLURM.R
R/installPackagesOnSSHWorkers.R
R/getJob.R
R/clusterFunctionsSGE.R
R/loadResult.R
R/doJob.R
R/testJob.R
R/waitForJobs.R
R/setJobFunction.R
R/clusterFunctionsSSH.R
R/checkIdValid.R
R/showClusterStatus.R
R/batchReduce.R
R/batchMapQuick.R
R/clusterFunctionsLSF.R
R/Resources.R
R/clusterFunctionsTorque.R
R/resetJobs.R
R/sweepRegistry.R
R/clusterFunctionsLocal.R
R/updateRegistry.R
R/batchMapResults.R
R/findJobs.R
R/getSSHWorkersInfo.R
R/grepLogs.R
R/batchMap.R
R/zzz.R
R/WorkerLinux.R
R/scheduleWorkerJobs.R
R/applyJobFunction.R
R/addJob.R
MD5
DESCRIPTION
man
man/makeClusterFunctionsLSF.Rd
man/loadResults.Rd
man/addRegistrySourceDirs.Rd
man/updateRegistry.Rd
man/dbCreateJobDefTable.Rd
man/getJobs.Rd
man/setJobNames.Rd
man/applyJobFunction.Rd
man/installPackagesOnSSHWorkers.Rd
man/makeClusterFunctionsMulticore.Rd
man/killJobs.Rd
man/makeRegistry.Rd
man/getErrorMessages.Rd
man/configuration.Rd
man/makeJob.Rd
man/makeClusterFunctionsSGE.Rd
man/batchExport.Rd
man/makeClusterFunctionsSLURM.Rd
man/batchMapResults.Rd
man/cfHandleUnknownSubmitError.Rd
man/loadResult.Rd
man/batchMap.Rd
man/showClusterStatus.Rd
man/sanitizePath.Rd
man/setConfig.Rd
man/waitForJobs.Rd
man/addRegistrySourceFiles.Rd
man/copyRequiredJobFiles.Rd
man/makeSubmitJobResult.Rd
man/showStatus.Rd
man/batchMapQuick.Rd
man/loadRegistry.Rd
man/reduceResults.Rd
man/batchReduceResults.Rd
man/filterResults.Rd
man/loadConfig.Rd
man/getJobInfo.Rd
man/grepLogs.Rd
man/makeClusterFunctions.Rd
man/submitJobs.Rd
man/makeClusterFunctionsInteractive.Rd
man/findState.Rd
man/cfKillBatchJob.Rd
man/findJobs.Rd
man/testJob.Rd
man/showLog.Rd
man/debugSSH.Rd
man/removeRegistrySourceDirs.Rd
man/getJobIds.Rd
man/getResources.Rd
man/batchReduce.Rd
man/makeClusterFunctionsTorque.Rd
man/addRegistryPackages.Rd
man/getLogFiles.Rd
man/debugMulticore.Rd
man/removeRegistryPackages.Rd
man/getJobParamDf.Rd
man/getJobResources.Rd
man/batchUnexport.Rd
man/getSSHWorkersInfo.Rd
man/sweepRegistry.Rd
man/batchExpandGrid.Rd
man/BatchJobs.Rd
man/getJobNr.Rd
man/makeClusterFunctionsSSH.Rd
man/getJob.Rd
man/resetJobs.Rd
man/cfBrewTemplate.Rd
man/sourceRegistryFiles.Rd
man/loadExports.Rd
man/setJobFunction.Rd
man/callFunctionOnSSHWorkers.Rd
man/dbGetJobs.Rd
man/getConfig.Rd
man/makeSSHWorker.Rd
man/removeRegistrySourceFiles.Rd
man/cfReadBrewTemplate.Rd
man/removeRegistry.Rd
man/makeClusterFunctionsLocal.Rd
LICENSE
BatchJobs documentation built on May 20, 2017, 4:34 a.m.

Questions? Problems? Suggestions? Tweet to @rdrrHQ or email at ian@mutexlabs.com.

Please suggest features or report bugs in the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.