In jeremyrcoyle/delayed: A Framework for Parallelizing Dependent Tasks

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "README-"
)

R/`delayed`

A framework for parallelizing dependent tasks

Author: Jeremy Coyle

What's `delayed`?

delayed is an R package that provides a framework for parallelizing dependent tasks in an efficient manner. It brings to R a subset of the functionality implemented in Python's Dask library. For details on how best to use delayed, please consult the package documentation and vignette online, or do so from within R.

Installation

For standard use, we recommend installing the package from CRAN via

install.packages("delayed")

Install the most recent stable release from GitHub via devtools:

devtools::install_github("tlverse/delayed")

Issues

If you encounter any bugs or have any specific feature requests, please file an issue.

Example

This minimal example shows how to use delayed to handle dependent computations via chaining of tasks:

library(delayed)

# delay a function that does a bit of math
mapfun <- function(x, y) {(x + y) / (x - y)}
delayed_mapfun <- delayed_fun(mapfun)

set.seed(14765)
library(future)
plan(multicore, workers = 2)
const <- 7

# re-define the delayed object from above
delayed_norm <- delayed(rnorm(n = const))
delayed_pois <- delayed(rpois(n = const, lambda = const))
chained_norm_pois <- delayed_mapfun(delayed_norm, delayed_pois)

# compute it using the future plan (multicore with 2 cores)
chained_norm_pois$compute(nworkers = 2, verbose = TRUE)

Remark: In the above, the delayed computation is carried out in parallel using the framework offered by the excellent future package and its associated ecosystem.