restbatch: A Simple 'RESTful' Web Service to Schedule R Batch Jobs

Description Usage Arguments Details Value Fields Methods Examples

Create/Restore a client-side task proxy

1
2
3

new_task2(fun, ..., task_name = "Noname", globals = TRUE, env = parent.frame())

restore_task2(task_name)

`fun, ...`	function to apply and additional parameters to be passed to `batchMap` function
`task_name`	a readable name for the task; default is `'Noname'`.
`globals`	variables to export to the computing processes; see details.
`env`	environment where the `'globals'` are defined

A task proxy manages a folder at task_dir containing all the information necessary to perform the task. A task is a collection of "jobs" (see its definition in batchMap). The evaluation of the jobs is always in different processes with the current R session, sometimes even different computers.

Sometimes the function fun may contains external objects that need to be exported. For example, function(x){ a+x } has a variable a that might be defined in somewhere else. It will raise errors if we simply execute the function. Therefore, the task requires exporting the variable a before executing jobs. This task could be done automatically or manually. If globals is TRUE, then all the external variables including the packages needed will be automatically detected and exported. If globals is a character vector, then only the variables indicated will be correctly exported If globals is FALSE, then you need to manually specify them in task$set_globals method

If the task does not exist, the returns NULL, otherwise returns a locked environment proxy that wraps batch task with the following fields and methods.

task_name: task ID, a combination of user ID, task readable name, and a 16-character random string
readable_name: a friendly name of the task
task_dir: task directory where the jobs are stored
task_root: root directory where all tasks are stored
reg: 'batchtools' registry that reads batch job status
njobs: total number of jobs in the task
protocol, host, port, path_*: server to be submitted to, used by 'submit()' method
submitted: whether the task has been submitted or not
submitted_to: if the task has been submitted, then where
results: the result of the task, should almost surely use collect() instead
collected: whether the results has been loaded from task directory into the memory

submit: submit or re-submit a task. If 'pack' is true, then the task files will be archived before sending to the server. This option is required when server runs remotely.
resolved: check whether the task has been resolved, a combination of server_status() and local_status(). Will raise errors if server if not running
locally_resolved: check whether task is resolved locally without query the server. Only used to check local files. It's possible that server has finished but locally not resolved. See download() to synchronize local files with the finished tasks on the server
server_status: get task status from servers
local_status: get task status from servers
download: download the finished tasks from the server and overwrite the local files. Raises error if the task is not finished remotely. Always check server status before downloading.
collect: collect results. It checks the status both locally and remotely; download the task if finished; load up the results. This function will block the session waiting for results.
zip: create an archive that stores the task
clear_registry: clear the registry, usually used to re-use the task or to re-run the task
reload_registry: reload 'reg' registry under read-only or writable modes. If the task has been submitted, please load in read-only mode to avoid file corrects
remove: remove the whole task from the hard drive

task <- new_task2(function(...){
  list(...)
}, x = 1:3, y = 1:6)

# Not submitted
task

# save task name to same other places
task_name <- task$task_name

# submit the task
if(interactive()){

  ensure_server()

  task$submit()

  task

  # once finished
  kill_server()

}

# Remove the task. You can always restore with task_name

rm(task)

task_restored <- restore_task2(task_name = task_name)

if(task_restored$submitted && task_restored$resolved()){
  task_restored$collect()
}

# clean up, remove all task files
task_restored$remove()