slurm_call: Execution of a single function call on the Slurm cluster

In rslurm: Submit R Calculations to a 'Slurm' Cluster

Execution of a single function call on the Slurm cluster

Description

Use slurm_call to perform a single function evaluation a the Slurm cluster.

Usage

slurm_call(
  f,
  params = list(),
  jobname = NA,
  global_objects = NULL,
  add_objects = NULL,
  pkgs = rev(.packages()),
  libPaths = NULL,
  rscript_path = NULL,
  r_template = NULL,
  sh_template = NULL,
  slurm_options = list(),
  submit = TRUE
)

Arguments

f	Any R function.
params	A named list of parameters to pass to f.
jobname	The name of the Slurm job; if NA, it is assigned a random name of the form "slr####".
global_objects	A character vector containing the name of R objects to be saved in a .RData file and loaded on each cluster node prior to calling f.
add_objects	Older deprecated name of global_objects, retained for backwards compatibility.
pkgs	A character vector containing the names of packages that must be loaded on each cluster node. By default, it includes all packages loaded by the user when slurm_call is called.
libPaths	A character vector describing the location of additional R library trees to search through, or NULL. The default value of NULL corresponds to libraries returned by .libPaths() on a cluster node. Non-existent library trees are silently ignored.
rscript_path	The location of the Rscript command. If not specified, defaults to the location of Rscript within the R installation being run.
r_template	The path to the template file for the R script run on each node. If NULL, uses the default template "rslurm/templates/slurm_run_single_R.txt".
sh_template	The path to the template file for the sbatch submission script. If NULL, uses the default template "rslurm/templates/submit_single_sh.txt".
slurm_options	A named list of options recognized by sbatch; see Details below for more information.
submit	Whether or not to submit the job to the cluster with sbatch; see Details below for more information.

Details

This function creates a temporary folder ("_rslurm_[jobname]") in the current directory, holding .RData and .RDS data files, the R script to run and the Bash submission script generated for the Slurm job.

The names of any other R objects (besides params) that f needs to access should be listed in the global_objects argument.

Use slurm_options to set any option recognized by sbatch, e.g. slurm_options = list(time = "1:00:00", share = TRUE). See http://slurm.schedmd.com/sbatch.html for details on possible options. Note that full names must be used (e.g. "time" rather than "t") and that flags (such as "share") must be specified as TRUE. The "job-name", "ntasks" and "output" options are already determined by slurm_call and should not be manually set.

When processing the computation job, the Slurm cluster will output two files in the temporary folder: one with the return value of the function ("results_0.RDS") and one containing any console or error output produced by R ("slurm_[node_id].out").

If submit = TRUE, the job is sent to the cluster and a confirmation message (or error) is output to the console. If submit = FALSE, a message indicates the location of the saved data and script files; the job can be submitted manually by running the shell command sbatch submit.sh from that directory.

After sending the job to the Slurm cluster, slurm_call returns a slurm_job object which can be used to cancel the job, get the job status or output, and delete the temporary files associated with it. See the description of the related functions for more details.

Value

A slurm_job object containing the jobname and the number of nodes effectively used.

See Also

slurm_apply to parallelize a function over a parameter set.

cancel_slurm, cleanup_files, get_slurm_out and get_job_status which use the output of this function.

rslurm documentation built on March 7, 2023, 5:37 p.m.