Task queues for R, implemented using Redis.
Create an rrq_controller
object
obj <- rrq::rrq_controller$new("rrq:readme")
Submit work to the queue:
t <- obj$enqueue(runif(10))
t
#> [1] "1604b9d2665fc935c5f432b0d7da4889"
Query task process:
obj$task_status(t)
#> 1604b9d2665fc935c5f432b0d7da4889
#> "PENDING"
Run tasks on workers in the background
rrq::rrq_worker_spawn(obj)
#> Spawning 1 worker with prefix hexagonal_xenurine
#> [1] "hexagonal_xenurine_1"
Collect task results when complete
obj$task_wait(t)
#> [1] 0.21934439 0.47012520 0.57837978 0.17550807 0.07085051 0.04189457
#> [7] 0.43439891 0.79186554 0.82606661 0.86208847
Or try and retrieve them regardless of if they are complete
obj$task_result(t)
#> [1] 0.21934439 0.47012520 0.57837978 0.17550807 0.07085051 0.04189457
#> [7] 0.43439891 0.79186554 0.82606661 0.86208847
Query what workers have done
obj$worker_log_tail(n = Inf)
#> worker_id time command
#> 1 hexagonal_xenurine_1 1678370725 ALIVE
#> 2 hexagonal_xenurine_1 1678370725 TASK_START
#> 3 hexagonal_xenurine_1 1678370726 TASK_COMPLETE
#> message
#> 1
#> 2 1604b9d2665fc935c5f432b0d7da4889
#> 3 1604b9d2665fc935c5f432b0d7da4889
For more information, see vignette("rrq")
Install from the mrc-ide package repository:
drat:::add("mrc-ide")
install.packages("rrq")
Alternatively, install with remotes
:
remotes::install_github("mrc-ide/rrq", upgrade = FALSE)
To test, we need a redis server that can be automatically connected to using the redux
defaults. This is satisfied if you have an unauthenticated redis server running on localhost, otherwise you should update the environment variable REDIS_URL
to point at a redis server. Do not use a production server, as the package will create and delete a lot of keys.
A suitable redis server can be started using docker with
./scripts/redis start
(and stopped with ./scripts/redis stop
)
MIT © Imperial College of Science, Technology and Medicine
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.