View source: R/queue_didehpc.R
queue_didehpc | R Documentation |
Create a queue object. This is an R6::R6Class object which you interact with by calling "methods" which are described below, and on the help page for queuer::queue_base, from which this derives.
queue_didehpc(
context,
config = didehpc_config(),
root = NULL,
initialise = TRUE,
provision = NULL,
login = NULL
)
context |
A context |
config |
Optional dide configuration information. |
root |
A root directory, not usually needed |
initialise |
Passed through to the base queue. If you set
this to |
provision |
A provisioning strategy to use. Options are
|
login |
Logical, indicating if we should immediately
login. If |
queuer::queue_base
-> queue_didehpc
config
Your didehpc_config()
for this queue.
Do not change this after queue creation as changes may not
take effect as expected.
client
A web_client object used to communicate with the web portal. See the help page for its documentation, but you will typically not need to interact with this.
queuer::queue_base$enqueue()
queuer::queue_base$enqueue_()
queuer::queue_base$enqueue_bulk()
queuer::queue_base$initialize_context()
queuer::queue_base$lapply()
queuer::queue_base$mapply()
queuer::queue_base$task_bundle_get()
queuer::queue_base$task_bundle_info()
queuer::queue_base$task_bundle_list()
queuer::queue_base$task_bundle_retry_failed()
queuer::queue_base$task_delete()
queuer::queue_base$task_get()
queuer::queue_base$task_list()
queuer::queue_base$task_result()
queuer::queue_base$task_retry_failed()
queuer::queue_base$task_status()
queuer::queue_base$task_times()
new()
Constructor
queue_didehpc_$new( context, config, root, initialise, provision, login, client = NULL )
context, config, root, initialise, provision, login
See above
client
A web_client object, primarily useful for testing the package
login()
Log onto the web portal. This will be called automatically at either when creating the object (by default) or when you make your first request to the portal. However, you can call this to refresh the session too.
queue_didehpc_$login(refresh = TRUE)
refresh
Logical, indicating if we should try logging on again, even if it looks like we already have. This will refresh the session, which is typically what you want to do.
cluster_load()
Report on the overall cluster usage
queue_didehpc_$cluster_load(cluster = NULL, nodes = TRUE)
cluster
Cluster to show; if TRUE
show the entire cluster
(via load_overall
), if NULL
defaults to the value
config$cluster
nodes
Show the individual nodes when printing
reconcile()
Attempt to reconcile any differences in task state
between our database and the HPC queue. This is needed when
tasks have crashed, or something otherwise bad has happened
and you have tasks stuck in PENDING
or RUNNING
that are
clearly not happy. This function does not play well with workers and
you should not use it if using them.
queue_didehpc_$reconcile(task_ids = NULL)
task_ids
A vector of tasks to check
submit()
Submit a task to the queue. Ordinarily you do not call
this directly, it is called by the $enqueue()
method of
queuer::queue_base when you create a task. However, you can
use this to resubmit a task that has failed if you think it will
run successfully a second time (e.g., because you cancelled it
the first time around).
queue_didehpc_$submit(task_ids, names = NULL, depends_on = NULL)
task_ids
A vector of task identifiers to submit.
names
Optional names for the tasks.
depends_on
Optional vector of dependencies, named by task id
submit_workers()
Submit workers to the queue. This only works if
use_rrq
or use_workers
is TRUE
in your configuration.
See vignette("workers")
for more information.
queue_didehpc_$submit_workers(n, timeout = 600, progress = NULL)
n
The number of workers to submit
timeout
The time to wait, in seconds, for all workers to come online. An error will be thrown if this time is exceeded.
progress
Logical, indicating if a progress bar should be printed while waiting for workers.
stop_workers()
Stop workers running on the cluster. See
vignette("workers")
for more information. By default
workers will timeout after 10 minutes of inactivity.
queue_didehpc_$stop_workers(worker_ids = NULL)
worker_ids
Vector of worker names to try and stop. By default all workers are stopped.
rrq_controller()
Return an rrq::rrq_controller object, if you have
set use_rrq
or use_workers
in your configuration. This is
a lightweight queue using your workers which is typically much
faster than submitting via $enqueue()
. See vignette("workers")
for more information.
queue_didehpc_$rrq_controller()
unsubmit()
Unsubmit tasks from the cluster. This removes the tasks from the queue if they have not been started yet, and stops them if currently running. It will have no effect if the tasks are completed (successfully or errored)
queue_didehpc_$unsubmit(task_ids)
task_ids
Can be a task id (string), a vector of task ids, a task, a list of tasks, a bundle returned by enqueue_bulk, or a list of bundles.
dide_id()
Find the DIDE task id of your task. This is the number displayed in the web portal.
queue_didehpc_$dide_id(task_ids)
task_ids
Vector of task identifiers to look up
dide_log()
Return the pre-context log of a task. Use this to find
out what has happened to a task that has unexpectedly failed, but
for which $log()
is uninformative.
queue_didehpc_$dide_log(task_id)
task_id
A single task id to check
provision_context()
Provision your context for running on the cluster.
This sets up the remote set of packages that your tasks will use.
See vignette("packages")
for more information.
queue_didehpc_$provision_context( policy = "verylazy", dryrun = FALSE, quiet = FALSE, show_progress = NULL, show_log = TRUE )
policy
The installation policy to use, as interpreted by
pkgdepends::pkg_solution
- so this should be verylazy
/lazy
(install missing packages but don't upgrade unless needed) or
upgrade
(upgrade packages as possible). In addition you can
also use later
which does nothing, or fake
which pretends
that it ran the provisioning. See vignette("packages")
for
details on these options.
dryrun
Do a dry run installation locally - this just checks that your requested set of packages is plausible, but does this without submitting a cluster job so it may be faster.
quiet
Logical, controls printing of informative messages
show_progress
Logical, controls printing of a spinning progress bar
show_log
Logical, controls printing of the log from the cluster
install_packages()
Install packages on the cluster. This can be used to
more directly install packages on the cluster than the
$provision_context
method that you would typically use.
See vignette("packages")
for more information.
queue_didehpc_$install_packages( packages, repos = NULL, policy = "lazy", dryrun = FALSE, show_progress = NULL, show_log = TRUE )
packages
A character vector of packages to install. These
can be names of CRAN packages or GitHub references etc; see
pkgdepends::new_pkg_installation_proposal()
and
vignette("packages")
for more details
repos
A character vector of repositories to use when installing. A suitable CRAN repo will be added if not detected.
policy
The installation policy to use, as interpreted by
pkgdepends::pkg_solution
- so this should be lazy
(install missing packages but don't upgrade unless needed) or
upgrade
(upgrade packages as possible). In addition you can
also use later
which does nothing, or fake
which pretends
that it ran the provisioning. See vignette("packages")
for
details on these options.
dryrun
Do a dry run installation locally - this just checks that your requested set of packages is plausible, but does this without submitting a cluster job so it may be faster.
show_progress
Logical, controls printing of a spinning progress bar
show_log
Logical, controls printing of the log from the cluster
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.