BatchJobs: Batch Computing with R

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF, SLURM and Sun Grid Engine. Multicore and SSH systems are also supported. For further details see the project web page.

AuthorBernd Bischl <bernd_bischl@gmx.net>, Michel Lang <michellang@gmail.com>, Henrik Bengtsson <henrikb@braju.com>
Date of publication2015-03-18 18:01:21
MaintainerBernd Bischl <bernd_bischl@gmx.net>
LicenseBSD_2_clause + file LICENSE
Version1.6
https://github.com/tudo-r/BatchJobs

View on CRAN

Man pages

addRegistryPackages: Add packages to registry.

addRegistrySourceDirs: Add source dirs to registry.

addRegistrySourceFiles: Add source files to registry.

applyJobFunction: applyJobFunction ONLY FOR INTERNAL USAGE.

batchExpandGrid: Map function over all combinations.

batchExport: Export R object to be available on the slaves.

BatchJobs: The BatchJobs package

batchMap: Maps a function over lists or vectors, adding jobs to a...

batchMapQuick: Combination of makeRegistry, batchMap and submitJobs.

batchMapResults: Maps a function over the results of a registry by using...

batchReduce: Reduces via a binary function over a list adding jobs to a...

batchReduceResults: Reduces results via a binary function and adds jobs for this...

batchUnexport: Unload exported R objects.

callFunctionOnSSHWorkers: Call an arbitrary function on specified SSH workers.

cfBrewTemplate: Cluster functions helper: Brew your template into a job...

cfHandleUnknownSubmitError: Cluster functions helper: Handle an unknown error during job...

cfKillBatchJob: Cluster functions helper: Kill a batch job via OS command

cfReadBrewTemplate: Cluster functions helper: Read in your brew template file.

configuration: BatchJobs configuration.

copyRequiredJobFiles: ONLY FOR INTERNAL USAGE.

dbCreateJobDefTable: ONLY FOR INTERNAL USAGE.

dbGetJobs: ONLY FOR INTERNAL USAGE.

debugMulticore: Helper function to debug multicore mode.

debugSSH: Helper function to debug SSH mode.

filterResults: Find all results where a specific condition is true.

findJobs: Finds ids of jobs that match a query.

findState: Find jobs depending on computional state.

getConfig: Returns a list of BatchJobs configuration settings

getErrorMessages: Get error messages of jobs.

getJob: Get job from registry by id.

getJobIds: Get ids of jobs in registry.

getJobInfo: Get computational information of jobs.

getJobNr: Get number of jobs in registry.

getJobParamDf: Returns parameters for all jobs as the rows of a data.frame.

getJobResources: Function to get the resources that were submitted for some...

getJobs: Get jobs from registry by id.

getLogFiles: Get log file paths for jobs.

getResources: Function to get job resources in job function.

getSSHWorkersInfo: Print and return R installation and other information for SSH...

grepLogs: Grep log files for a pattern.

installPackagesOnSSHWorkers: Install packages on SSH workers.

killJobs: Kill some jobs on the batch system.

loadConfig: Load a specific configuration file.

loadExports: Load exported R data objects.

loadRegistry: Load a previously saved registry.

loadResult: Loads a specific result file.

loadResults: Loads result files for id vector.

makeClusterFunctions: Create a ClusterFuntions object.

makeClusterFunctionsInteractive: Create cluster functions for sequential execution in same...

makeClusterFunctionsLocal: Create cluster functions for sequential execution on local...

makeClusterFunctionsLSF: Create cluster functions for LSF systems.

makeClusterFunctionsMulticore: Use multiple cores on local Linux machine to spawn parallel...

makeClusterFunctionsSGE: Create cluster functions for Sun Grid Engine systems.

makeClusterFunctionsSLURM: Create cluster functions for SLURM-based systems.

makeClusterFunctionsSSH: Create an SSH cluster to execute jobs.

makeClusterFunctionsTorque: Create cluster functions for torque-based systems.

makeJob: Creates a job description.

makeRegistry: Construct a registry object.

makeSSHWorker: Create SSH worker for SSH cluster functions.

makeSubmitJobResult: Create a SubmitJobResult object.

reduceResults: Reduce results from result directory.

removeRegistry: Remove a registry object.

removeRegistryPackages: Remove packages from registry.

removeRegistrySourceDirs: Remove packages from registry.

removeRegistrySourceFiles: Remove source files from registry.

resetJobs: Reset computational state of jobs.

sanitizePath: Sanitize a path

setConfig: Set and overwrite configuration settings

setJobFunction: Sets the job function for already existing jobs.

setJobNames: Set job names.

showClusterStatus: Show information about available computational resources on...

showLog: Display the contents of a log file.

showStatus: Retrieve or show status information about jobs.

sourceRegistryFiles: Source registry files

submitJobs: Submit jobs or chunks of jobs to batch system via cluster...

sweepRegistry: Sweep obsolete files from the file system.

testJob: Tests a job by running it with Rscript in a new process.

updateRegistry: ONLY FOR INTERNAL USAGE.

waitForJobs: Wait for termination of jobs on the batch system.

Files in this package

BatchJobs
BatchJobs/inst
BatchJobs/inst/CITATION
BatchJobs/inst/bin
BatchJobs/inst/bin/linux-helper
BatchJobs/inst/etc
BatchJobs/inst/etc/BatchJobs_global_config.R
BatchJobs/tests
BatchJobs/tests/run-all.R
BatchJobs/tests/testthat
BatchJobs/tests/testthat/test_sweepRegistry.R
BatchJobs/tests/testthat/test_export.R
BatchJobs/tests/testthat/helpers.R
BatchJobs/tests/testthat/test_reduceResults.R
BatchJobs/tests/testthat/test_getJobInfo.R
BatchJobs/tests/testthat/test_sanitizePath.R
BatchJobs/tests/testthat/test_testJob.R
BatchJobs/tests/testthat/test_filterResults.R
BatchJobs/tests/testthat/test_getJobParamDf.R
BatchJobs/tests/testthat/test_batchMapQuick.R
BatchJobs/tests/testthat/test_loadResults.R
BatchJobs/tests/testthat/test_submitJobs.R
BatchJobs/tests/testthat/test_resources.R
BatchJobs/tests/testthat/test_resetJobs.R
BatchJobs/tests/testthat/test_getLogFiles.R
BatchJobs/tests/testthat/test_waitForJobs.R
BatchJobs/tests/testthat/test_batchMap.R
BatchJobs/tests/testthat/test_setJobFunction.R
BatchJobs/tests/testthat/test_findJobs.R
BatchJobs/tests/testthat/test_showStatus.R
BatchJobs/tests/testthat/test_batchReduce.R
BatchJobs/tests/testthat/test_findStatus.R
BatchJobs/tests/testthat/test_batchMapResults.R
BatchJobs/tests/testthat/test_batchReduceResults.R
BatchJobs/tests/testthat/test_sourceRegistryFiles.R
BatchJobs/tests/testthat/test_zzz_cleanup.R
BatchJobs/tests/testthat/test_getJob.R
BatchJobs/tests/testthat/test_database.R
BatchJobs/tests/testthat/test_sqlquotes.R
BatchJobs/tests/testthat/test_batchExpandGrid.R
BatchJobs/tests/testthat/test_packages.R
BatchJobs/tests/testthat/test_makeRegistry.R
BatchJobs/tests/testthat/test_grepLogs.R
BatchJobs/tests/testthat/test_doJob.R
BatchJobs/NAMESPACE
BatchJobs/NEWS
BatchJobs/R
BatchJobs/R/getJobInfo.R BatchJobs/R/buffer.R BatchJobs/R/clusterFunctionsInteractive.R BatchJobs/R/clusterFunctions.R BatchJobs/R/Registry.R BatchJobs/R/filterResults.R BatchJobs/R/reduceResults.R BatchJobs/R/killJobs.R BatchJobs/R/database.R BatchJobs/R/writeRscripts.R BatchJobs/R/callFunctionOnSSHWorkers.R BatchJobs/R/getJobParamDf.R BatchJobs/R/helpers.R BatchJobs/R/Worker.R BatchJobs/R/Job.R BatchJobs/R/conf.R BatchJobs/R/debugMulticore.R BatchJobs/R/clusterFunctionsHelpers.R BatchJobs/R/showLog.R BatchJobs/R/debugSSH.R BatchJobs/R/batchApply.R BatchJobs/R/batchExpandGrid.R BatchJobs/R/filenames.R BatchJobs/R/Packages.R BatchJobs/R/getErrorMessages.R BatchJobs/R/loadResults.R BatchJobs/R/setJobNames.R BatchJobs/R/sanitizePath.R BatchJobs/R/findStatus.R BatchJobs/R/submitJobs.R BatchJobs/R/batchReduceResults.R BatchJobs/R/clusterFunctionsMulticore.R BatchJobs/R/Exports.R BatchJobs/R/syncRegistry.R BatchJobs/R/getLogFiles.R BatchJobs/R/sourceRegistryFiles.R BatchJobs/R/showStatus.R BatchJobs/R/sendMail.R BatchJobs/R/clusterFunctionsSLURM.R BatchJobs/R/installPackagesOnSSHWorkers.R BatchJobs/R/getJob.R BatchJobs/R/clusterFunctionsSGE.R BatchJobs/R/loadResult.R BatchJobs/R/doJob.R BatchJobs/R/testJob.R BatchJobs/R/waitForJobs.R BatchJobs/R/setJobFunction.R BatchJobs/R/clusterFunctionsSSH.R BatchJobs/R/checkIdValid.R BatchJobs/R/showClusterStatus.R BatchJobs/R/batchReduce.R BatchJobs/R/batchMapQuick.R BatchJobs/R/clusterFunctionsLSF.R BatchJobs/R/Resources.R BatchJobs/R/clusterFunctionsTorque.R BatchJobs/R/resetJobs.R BatchJobs/R/sweepRegistry.R BatchJobs/R/clusterFunctionsLocal.R BatchJobs/R/updateRegistry.R BatchJobs/R/batchMapResults.R BatchJobs/R/findJobs.R BatchJobs/R/getSSHWorkersInfo.R BatchJobs/R/grepLogs.R BatchJobs/R/batchMap.R BatchJobs/R/zzz.R BatchJobs/R/WorkerLinux.R BatchJobs/R/scheduleWorkerJobs.R BatchJobs/R/applyJobFunction.R BatchJobs/R/addJob.R
BatchJobs/MD5
BatchJobs/DESCRIPTION
BatchJobs/man
BatchJobs/man/makeClusterFunctionsLSF.Rd BatchJobs/man/loadResults.Rd BatchJobs/man/addRegistrySourceDirs.Rd BatchJobs/man/updateRegistry.Rd BatchJobs/man/dbCreateJobDefTable.Rd BatchJobs/man/getJobs.Rd BatchJobs/man/setJobNames.Rd BatchJobs/man/applyJobFunction.Rd BatchJobs/man/installPackagesOnSSHWorkers.Rd BatchJobs/man/makeClusterFunctionsMulticore.Rd BatchJobs/man/killJobs.Rd BatchJobs/man/makeRegistry.Rd BatchJobs/man/getErrorMessages.Rd BatchJobs/man/configuration.Rd BatchJobs/man/makeJob.Rd BatchJobs/man/makeClusterFunctionsSGE.Rd BatchJobs/man/batchExport.Rd BatchJobs/man/makeClusterFunctionsSLURM.Rd BatchJobs/man/batchMapResults.Rd BatchJobs/man/cfHandleUnknownSubmitError.Rd BatchJobs/man/loadResult.Rd BatchJobs/man/batchMap.Rd BatchJobs/man/showClusterStatus.Rd BatchJobs/man/sanitizePath.Rd BatchJobs/man/setConfig.Rd BatchJobs/man/waitForJobs.Rd BatchJobs/man/addRegistrySourceFiles.Rd BatchJobs/man/copyRequiredJobFiles.Rd BatchJobs/man/makeSubmitJobResult.Rd BatchJobs/man/showStatus.Rd BatchJobs/man/batchMapQuick.Rd BatchJobs/man/loadRegistry.Rd BatchJobs/man/reduceResults.Rd BatchJobs/man/batchReduceResults.Rd BatchJobs/man/filterResults.Rd BatchJobs/man/loadConfig.Rd BatchJobs/man/getJobInfo.Rd BatchJobs/man/grepLogs.Rd BatchJobs/man/makeClusterFunctions.Rd BatchJobs/man/submitJobs.Rd BatchJobs/man/makeClusterFunctionsInteractive.Rd BatchJobs/man/findState.Rd BatchJobs/man/cfKillBatchJob.Rd BatchJobs/man/findJobs.Rd BatchJobs/man/testJob.Rd BatchJobs/man/showLog.Rd BatchJobs/man/debugSSH.Rd BatchJobs/man/removeRegistrySourceDirs.Rd BatchJobs/man/getJobIds.Rd BatchJobs/man/getResources.Rd BatchJobs/man/batchReduce.Rd BatchJobs/man/makeClusterFunctionsTorque.Rd BatchJobs/man/addRegistryPackages.Rd BatchJobs/man/getLogFiles.Rd BatchJobs/man/debugMulticore.Rd BatchJobs/man/removeRegistryPackages.Rd BatchJobs/man/getJobParamDf.Rd BatchJobs/man/getJobResources.Rd BatchJobs/man/batchUnexport.Rd BatchJobs/man/getSSHWorkersInfo.Rd BatchJobs/man/sweepRegistry.Rd BatchJobs/man/batchExpandGrid.Rd BatchJobs/man/BatchJobs.Rd BatchJobs/man/getJobNr.Rd BatchJobs/man/makeClusterFunctionsSSH.Rd BatchJobs/man/getJob.Rd BatchJobs/man/resetJobs.Rd BatchJobs/man/cfBrewTemplate.Rd BatchJobs/man/sourceRegistryFiles.Rd BatchJobs/man/loadExports.Rd BatchJobs/man/setJobFunction.Rd BatchJobs/man/callFunctionOnSSHWorkers.Rd BatchJobs/man/dbGetJobs.Rd BatchJobs/man/getConfig.Rd BatchJobs/man/makeSSHWorker.Rd BatchJobs/man/removeRegistrySourceFiles.Rd BatchJobs/man/cfReadBrewTemplate.Rd BatchJobs/man/removeRegistry.Rd BatchJobs/man/makeClusterFunctionsLocal.Rd
BatchJobs/LICENSE

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

All documentation is copyright its authors; we didn't write any of that.