runbg_bwfor_slurm: Run an R expression on the bwForCluster via sshpass and slurm

In dkaschek/dMod: Dynamic Modeling and Parameter Estimation in ODE Models

Run an R expression on the bwForCluster via sshpass and slurm

Description

Generate an R code of the expression that is copied via scp to the bwForCluster. Then collect the results. ssh-key not needed. Password can be provided via an additional argument. sshpass needs to be installed on your local machine.

Usage

runbg_bwfor_slurm(
  ...,
  machine,
  filename = NULL,
  nodes = 1,
  cores = 1,
  partition = "single",
  walltime = "01:00:00",
  input = ls(.GlobalEnv),
  compile = TRUE,
  recover = F,
  password = "'begin__end'"
)

Arguments

...	Some R code
machine	e.g. [e-mail]
filename	Character, defining the filename of the temporary file. Random file name ist chosen if NULL. Must not contain the string "Minus".
nodes	Number of nodes, e.g. 10
cores	Number of cores, e.g. 16
partition	character, the partition where to start the job
walltime	estimated runtime in the format hh:mm:ss, e.g. 01:30:00. Jobs with a walltime up to 30 min are sent to a quick queue. When the walltime is exceeded, all jobs are automatically killed by the queue.
input	Character vector, the objects in the workspace that are stored into an R data file and copied to the remove machine.
compile	Logical. If TRUE, C files are copied and compiled on the remote machine. Otherwise, the .so files are copied.
recover	Logical, If TRUE, the scripts will not be started again. Can be used to get back the check and get functions for an already started process, e.g. after local session has aborted.
password	Your ssh password in plain text (yes, no joke unfortunately), the password is handed over to sshpass for automatic login on the cluster. If NULL, the standard ssh/scp is used and you will be asked for your password multiple times while uploading the scripts.

Details

runbg() generates a workspace from the input argument and copies the workspace and all C files or .so files to the remote machines via scp. This will only work if *an ssh-key had been generated and added to the authorized keys on the remote machine*. The code snippet, i.e. the ... argument, can include several intermediate results but only the last call which is not redirected into a variable is returned via the variable .runbgOutput, see example below.

Value

List of functions check(), get() and purge(). check() checks, if the result is ready. get() copies the result file to the working directory and loads it into the workspace as an object called .runbgOutput. This object is a list named according to the machines that contains the results returned by each machine. purge() deletes the temporary folder from the working directory and the remote machines.

Examples

## Not run: 
out_job1 <- runbg({
   mstrust(obj, center, fits = 10, cores = 2)
 }, 
 machine = "bwfor", nodes = 2, cores = "2:best", 
 walltime = "00:01:00", 
 filename = "job1")
out_job1$check()          
out_job1$get()
out_job1$purge()
result <- .runbgOutput
print(result)

## End(Not run)

dkaschek/dMod documentation built on April 23, 2024, 5:18 p.m.