In wlandau/drake.future.lapply.staged: Staged Parallelism for the Drake R Package

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
library(drake.future.lapply.staged)

Staged parallelism for the drake R package

With staged parallelism, drake partitions the dependency graph into stages of conditionally independent targets and processes each stage with semi-transient parallel workers. This functionality is already deprecated, and it will be removed at some point later on.

Installation

library(remotes)
install_github("ropensci/drake")
install_github("wlandau/drake.future.lapply.staged")

Usage

We begin with a drake project.

library(drake.future.lapply.staged)
plan <- drake_plan(x = rnorm(100), y = mean(x), z = median(x))

plan

First, create a future plan. See the future README and future.batchtools README for guidance, and consult tables here and here for options for your plan.

library(future)
plan(multiprocess)

Next, run your drake project.

library(drake.future.lapply.staged)
make(plan, parallelism = backend_future_lapply_staged, jobs = 2)

unlink(".drake", recursive = TRUE)

wlandau/drake.future.lapply.staged documentation built on May 23, 2019, 5:08 p.m.