startLocalWorkers: Start one or more background R worker processes on the local...

In bwlewis/future.redis: An Elastic Distributed Computing Backend for the Future Package

Start one or more background R worker processes on the local system.

Description

Use startLocalWorkers to start one or more future.redis R worker processes in the background. The worker processes are started on the local system using the worker function.

Usage

startLocalWorkers(
  n,
  queue = getOption("future.redis.queue", "{{session}}"),
  config = redis_config(),
  iter = Inf,
  linger = 10,
  log = NULL,
  Rbin = paste(R.home(component = "bin"), "R", sep = "/")
)

Arguments

n	number of workers to start.
queue	Redis task queue name.
config	Redis configuration (see redis_config)).
iter	Maximum number of tasks to acquire before exiting.
linger	in seconds, max time before system checks (including termination).
log	divert stdout and messages to log file.
Rbin	full path to the command-line R program.

Details

Running workers self-terminate after a linger interval if their task queue is deleted with the removeQ function, or if network communication with the Redis server encounters an error.

Value

NULL is invisibly returned.

See Also

redis_config, worker, removeQ

Examples

if (redux::redis_available()) {
## The example assumes that a Redis server is running on the local host
## and standard port.

# Register the redis plan on a specified task queue:
plan(redis, queue = "R jobs")

# Start some local R worker processes:
startLocalWorkers(n=2, queue="R jobs", linger=1)

# Alternatively, use the following to run the workers quietly without
# showing their output as they run:
# startLocalWorkers(n=2, queue="R jobs", linger=1, log="/dev/null")

# A function that returns a future (note the scope of N)
f <- \() future({4 * sum((runif(N) ^ 2 + runif(N) ^ 2) < 1) / N}, seed = TRUE)

# Run a simple sampling approximation of pi in parallel using  M * N points:
N <- 1e6  # samples per worker
M <- 10   # iterations
Reduce(sum, Map(value, replicate(M, f()))) / M

# Clean up
removeQ("R jobs")
}

bwlewis/future.redis documentation built on May 12, 2023, 1:52 a.m.