crew_controller_local: Create a controller with a local process launcher.
In wlandau/crew: A Distributed Worker Launcher Framework

crew_controller_local

R Documentation

Create a controller with a local process launcher.

Description

Create an R6 object to submit tasks and launch workers on local processes.

Usage

crew_controller_local(
  name = NULL,
  workers = 1L,
  host = "127.0.0.1",
  port = NULL,
  tls = crew::crew_tls(),
  tls_enable = NULL,
  tls_config = NULL,
  seconds_interval = 1,
  seconds_timeout = 60,
  seconds_launch = 30,
  seconds_idle = 300,
  seconds_wall = Inf,
  seconds_exit = NULL,
  retry_tasks = NULL,
  tasks_max = Inf,
  tasks_timers = 0L,
  reset_globals = TRUE,
  reset_packages = FALSE,
  reset_options = FALSE,
  garbage_collection = FALSE,
  crashes_error = NULL,
  launch_max = NULL,
  r_arguments = c("--no-save", "--no-restore"),
  crashes_max = 5L,
  backup = NULL,
  options_metrics = crew::crew_options_metrics(),
  options_local = crew::crew_options_local(),
  local_log_directory = NULL,
  local_log_join = NULL
)

Arguments

`name`	Deprecated on 2025-01-14 (`crew` version 0.10.2.9002).
`workers`	Deprecated on 2025-01-13 (`crew` version 0.10.2.9002).
`host`	IP address of the `mirai` client to send and receive tasks. If `NULL`, the host defaults to the local IP address.
`port`	TCP port to listen for the workers. If `NULL`, then an available ephemeral port is automatically chosen. Controllers running simultaneously on the same computer (as in a controller group) must not share the same TCP port.
`tls`	A TLS configuration object from `crew_tls()`.
`tls_enable`	Deprecated on 2023-09-15 in version 0.4.1. Use argument `tls` instead.
`tls_config`	Deprecated on 2023-09-15 in version 0.4.1. Use argument `tls` instead.
`seconds_interval`	Number of seconds between polling intervals waiting for certain internal synchronous operations to complete, such as checking `mirai::status()`
`seconds_timeout`	Number of seconds until timing out while waiting for certain synchronous operations to complete, such as checking `mirai::status()`.
`seconds_launch`	Seconds of startup time to allow. A worker is unconditionally assumed to be alive from the moment of its launch until `seconds_launch` seconds later. After `seconds_launch` seconds, the worker is only considered alive if it is actively connected to its assign websocket.
`seconds_idle`	Maximum number of seconds that a worker can idle since the completion of the last task. If exceeded, the worker exits. But the timer does not launch until `tasks_timers` tasks have completed. See the `idletime` argument of `mirai::daemon()`. `crew` does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, so please allow enough idle time for a new worker to be delegated a new task.
`seconds_wall`	Soft wall time in seconds. The timer does not launch until `tasks_timers` tasks have completed. See the `walltime` argument of `mirai::daemon()`.
`seconds_exit`	Deprecated on 2023-09-21 in version 0.5.0.9002. No longer necessary.
`retry_tasks`	Deprecated on 2025-01-13 (`crew` version 0.10.2.9002).
`tasks_max`	Maximum number of tasks that a worker will do before exiting. See the `maxtasks` argument of `mirai::daemon()`. `crew` does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, it is recommended to set `tasks_max` to a value greater than 1.
`tasks_timers`	Number of tasks to do before activating the timers for `seconds_idle` and `seconds_wall`. See the `timerstart` argument of `mirai::daemon()`.
`reset_globals`	`TRUE` to reset global environment variables between tasks, `FALSE` to leave them alone.
`reset_packages`	`TRUE` to unload any packages loaded during a task (runs between each task), `FALSE` to leave packages alone.
`reset_options`	`TRUE` to reset global options to their original state between each task, `FALSE` otherwise. It is recommended to only set `reset_options = TRUE` if `reset_packages` is also `TRUE` because packages sometimes rely on options they set at loading time.
`garbage_collection`	`TRUE` to run garbage collection between tasks, `FALSE` to skip.
`crashes_error`	Deprecated on 2025-01-13 (`crew` version 0.10.2.9002).
`launch_max`	Deprecated on 2024-11-04 (`crew` version 0.10.2.9002). Use `crashes_error` instead.
`r_arguments`	Optional character vector of command line arguments to pass to `Rscript` (non-Windows) or `Rscript.exe` (Windows) when starting a worker. Example: `r_arguments = c("--vanilla", "--max-connections=32")`.
`crashes_max`	In rare cases, a worker may exit unexpectedly before it completes its current task. If this happens, `pop()` returns a status of `"crash"` instead of `"error"` for the task. The controller does not automatically retry the task, but you can retry it manually by calling `push()` again and using the same task name as before. (However, `targets` pipelines running `crew` do automatically retry tasks whose workers crashed.) `crashes_max` is a non-negative integer, and it sets the maximum number of allowable consecutive crashes for a given task. If a task's worker crashes more than `crashes_max` times in a row, then `pop()` throws an error when it tries to return the results of the task.
`backup`	An optional `crew` controller object, or `NULL` to omit. If supplied, the `backup` controller runs any pushed tasks that have already reached `crashes_max` consecutive crashes. Using `backup`, you can create a chain of controllers with different levels of resources (such as worker memory and CPUs) so that a task that fails on one controller can retry using incrementally more powerful workers. All controllers in a backup chain should be part of the same controller group (see `crew_controller_group()`) so you can call the group-level `pop()` and `collect()` methods to make sure you get results regardless of which controller actually ended up running the task. Limitations of `backup`: * `crashes_max` needs to be positive in order for `backup` to be used. Otherwise, every task would always skip the current controller and go to `backup`. * `backup` cannot be a controller group. It must be an ordinary controller.
`options_metrics`	Either `NULL` to opt out of resource metric logging for workers, or an object from `crew_options_metrics()` to enable and configure resource metric logging for workers. For resource logging to run, the `autometric` R package version 0.1.0 or higher must be installed.
`options_local`	An object generated by `crew_options_local()` with options specific to the local controller.
`local_log_directory`	Deprecated on 2024-10-08. Use `options_local` instead.
`local_log_join`	Deprecated on 2024-10-08. Use `options_local` instead.

Examples

if (identical(Sys.getenv("CREW_EXAMPLES"), "true")) {
controller <- crew_controller_local()
controller$start()
controller$push(name = "task", command = sqrt(4))
controller$wait()
controller$pop()
controller$terminate()
}

wlandau/crew documentation built on Feb. 8, 2025, 10:12 a.m.