crew_launcher: Create an abstract launcher.
In wlandau/crew: A Distributed Worker Launcher Framework

crew_launcher

R Documentation

Create an abstract launcher.

Description

This function is useful for inheriting argument documentation in functions that create custom third-party launchers. See ⁠@inheritParams crew::crew_launcher⁠ in the source code file of crew_launcher_local().

Usage

crew_launcher(
  name = NULL,
  workers = 1L,
  seconds_interval = 1,
  seconds_timeout = 60,
  seconds_launch = 30,
  seconds_idle = 300,
  seconds_wall = Inf,
  seconds_exit = NULL,
  tasks_max = Inf,
  tasks_timers = 0L,
  reset_globals = TRUE,
  reset_packages = FALSE,
  reset_options = FALSE,
  garbage_collection = FALSE,
  crashes_error = NULL,
  launch_max = NULL,
  tls = crew::crew_tls(),
  processes = NULL,
  r_arguments = c("--no-save", "--no-restore"),
  options_metrics = crew::crew_options_metrics()
)

Arguments

`name`	Character string, name of the launcher. If the name is `NULL`, then a name is automatically generated when the launcher starts.
`workers`	Maximum number of workers to run concurrently when auto-scaling, excluding task retries and manual calls to `launch()`. Special workers allocated for task retries do not count towards this limit, so the number of workers running at a given time may exceed this maximum. A smaller number of workers may run if the number of executing tasks is smaller than the supplied value of the `workers` argument.
`seconds_interval`	Number of seconds between polling intervals waiting for certain internal synchronous operations to complete. In certain cases, exponential backoff is used with this argument passed to `seconds_max` in a `crew_throttle()` object.
`seconds_timeout`	Number of seconds until timing out while waiting for certain synchronous operations to complete, such as checking `mirai::status()`.
`seconds_launch`	Seconds of startup time to allow. A worker is unconditionally assumed to be alive from the moment of its launch until `seconds_launch` seconds later. After `seconds_launch` seconds, the worker is only considered alive if it is actively connected to its assign websocket.
`seconds_idle`	Maximum number of seconds that a worker can idle since the completion of the last task. If exceeded, the worker exits. But the timer does not launch until `tasks_timers` tasks have completed. See the `idletime` argument of `mirai::daemon()`. `crew` does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, so please allow enough idle time for a new worker to be delegated a new task.
`seconds_wall`	Soft wall time in seconds. The timer does not launch until `tasks_timers` tasks have completed. See the `walltime` argument of `mirai::daemon()`.
`seconds_exit`	Deprecated on 2023-09-21 in version 0.5.0.9002. No longer necessary.
`tasks_max`	Maximum number of tasks that a worker will do before exiting. See the `maxtasks` argument of `mirai::daemon()`. `crew` does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, it is recommended to set `tasks_max` to a value greater than 1.
`tasks_timers`	Number of tasks to do before activating the timers for `seconds_idle` and `seconds_wall`. See the `timerstart` argument of `mirai::daemon()`.
`reset_globals`	`TRUE` to reset global environment variables between tasks, `FALSE` to leave them alone.
`reset_packages`	`TRUE` to unload any packages loaded during a task (runs between each task), `FALSE` to leave packages alone.
`reset_options`	`TRUE` to reset global options to their original state between each task, `FALSE` otherwise. It is recommended to only set `reset_options = TRUE` if `reset_packages` is also `TRUE` because packages sometimes rely on options they set at loading time.
`garbage_collection`	`TRUE` to run garbage collection between tasks, `FALSE` to skip.
`crashes_error`	Deprecated on 2025-01-13 (`crew` version 0.10.2.9002).
`launch_max`	Deprecated on 2024-11-04 (`crew` version 0.10.2.9002). Use `crashes_error` instead.
`tls`	A TLS configuration object from `crew_tls()`.
`processes`	`NULL` or positive integer of length 1, number of local processes to launch to allow worker launches to happen asynchronously. If `NULL`, then no local processes are launched. If 1 or greater, then the launcher starts the processes on `start()` and ends them on `terminate()`. Plugins that may use these processes should run asynchronous calls using `launcher$async$eval()` and expect a `mirai` task object as the return value.
`r_arguments`	Optional character vector of command line arguments to pass to `Rscript` (non-Windows) or `Rscript.exe` (Windows) when starting a worker. Example: `r_arguments = c("--vanilla", "--max-connections=32")`.
`options_metrics`	Either `NULL` to opt out of resource metric logging for workers, or an object from `crew_options_metrics()` to enable and configure resource metric logging for workers. For resource logging to run, the `autometric` R package version 0.1.0 or higher must be installed.

Examples

if (identical(Sys.getenv("CREW_EXAMPLES"), "true")) {
client <- crew_client()
client$start()
launcher <- crew_launcher_local()
launcher$start(url = client$url, profile = client$profile)
launcher$launch()
task <- mirai::mirai("result", .compute = client$profile)
mirai::call_mirai_(task)
task$data
client$terminate()
}

wlandau/crew documentation built on Feb. 8, 2025, 10:12 a.m.