With staged parallelism, drake
partitions the dependency graph into stages of conditionally independent targets and processes each stage with semi-transient parallel workers. This functionality is already deprecated, and it will be removed at some point later on.
library(remotes)
install_github("ropensci/drake")
install_github("wlandau/drake.future.lapply.staged")
We begin with a drake
project.
library(drake.future.lapply.staged)
plan <- drake_plan(x = rnorm(100), y = mean(x), z = median(x))
plan
#> # A tibble: 3 x 2
#> target command
#> <chr> <chr>
#> 1 x rnorm(100)
#> 2 y mean(x)
#> 3 z median(x)
First, create a future
plan. See the future
README and future.batchtools
README for guidance, and consult tables here and here for options for your plan.
library(future)
plan(multiprocess)
Next, run your drake
project.
library(drake.future.lapply.staged)
make(plan, parallelism = backend_future_lapply_staged, jobs = 2)
#> Warning: `drake` can indeed accept a custom scheduler function for the
#> `parallelism` argument of `make()` but this is only for the sake of
#> experimentation and graceful deprecation. Your own custom schedulers may
#> cause surprising errors. Use at your own risk.
#> Warning: Staged parallelism for drake is deprecated and will be removed
#> eventually.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.