README.md
In mirai: Minimalist Async Evaluation Framework for R

mirai

Minimalist async evaluation framework for R. Lightweight parallel code execution and distributed computing. Designed for simplicity, a ‘mirai’ evaluates an R expression asynchronously, on local or network resources, resolving automatically upon completion. mirai() returns a ‘mirai’ object immediately. ‘mirai’ (未来みらい) is Japanese for ‘future’. Features efficient task scheduling, fast inter-process communications, and TLS over TCP/IP for remote connections. {mirai} has a tiny pure R code base, relying solely on nanonext, a high-performance binding for the ‘NNG’ (Nanomsg Next Gen) C library with zero package dependencies.

Install the latest release from CRAN:

install.packages("mirai")

or the development version from rOpenSci R-universe:

install.packages("mirai", repos = "https://shikokuchuo.r-universe.dev")

Use mirai() to evaluate an expression asynchronously in a separate, clean R process.

A ‘mirai’ object is returned immediately.

library(mirai)

m <- mirai(
  {
    res <- rnorm(x) + y ^ 2
    res / rev(res)
  },
  x = 11,
  y = runif(1)
)

m
#> < mirai >
#>  - $data for evaluated result

Above, all specified name = value pairs are passed through to the ‘mirai’.

The ‘mirai’ yields an ‘unresolved’ logical NA whilst the async operation is ongoing.

m$data
#> 'unresolved' logi NA

To check whether a mirai has resolved:

unresolved(m)
#> [1] FALSE

Upon completion, the ‘mirai’ resolves automatically to the evaluated result.

m$data
#>  [1] -0.4189294  0.4175360 -7.1428468 -0.3830076  1.6176787  1.0000000
#>  [7]  0.6181697 -2.6109140 -0.1400002  2.3950031 -2.3870373

Alternatively, explicitly call and wait for the result using call_mirai().

call_mirai(m)$data
#>  [1] -0.4189294  0.4175360 -7.1428468 -0.3830076  1.6176787  1.0000000
#>  [7]  0.6181697 -2.6109140 -0.1400002  2.3950031 -2.3870373

See the mirai vignette for full package functionality.

Key topics include:

Example use cases
Local daemons - persistent background processes
Distributed computing - remote daemons
Secure TLS connections

This may be accessed within R by:

vignette("mirai", package = "mirai")

{mirai} provides an alternative communications backend for R’s base ‘parallel’ package.

make_cluster() creates a ‘miraiCluster’, a cluster fully compatible with all ‘parallel’ functions such as:

parallel::clusterApply()
parallel::parLapply()
parallel::parLapplyLB()

doParallel can also register a ‘miraiCluster’ for use with the foreach package.

This functionality fulfils a request from R-Core at R Project Sprint 2023.

The crew package is a distributed worker-launcher extending {mirai} to different distributed computing platforms, from traditional clusters to cloud services.

crew.cluster is a plug-in that enables mirai-based workflows on traditional high-performance computing clusters using:

LFS
PBS/TORQUE
SGE
SLURM

targets, a Make-like pipeline tool for statistics and data science, has integrated and adopted crew as its default recommended high-performance computing backend.

{mirai} serves as a backend for enterprise asynchronous shiny or plumber applications.

A ‘mirai’ may be used interchangeably with a ‘promise’ by using the the promise pipe %...>%, or explictly by promises::as.promise(), allowing side-effects to be performed upon asynchronous resolution of a ‘mirai’.

Alternatively, crew also provides an interface that facilitates deploying {mirai} for shiny, and provides a Shiny vignette with tutorial and sample code for this purpose.

We would like to thank in particular:

William Landau, for being instrumental in shaping development of the package, from initiating the original request for persistent daemons, through to orchestrating robustness testing for the high performance computing requirements of crew and targets.

Henrik Bengtsson, for valuable and incisive insights leading to the interface accepting broader usage patterns.

Luke Tierney, R Core, for introducing R’s implementation of L’Ecuyer-CMRG streams, used to ensure statistical independence in parallel processing.

« Back to ToC

mirai website: https://shikokuchuo.net/mirai/ mirai on CRAN: https://cran.r-project.org/package=mirai

Listed in CRAN Task View: - High Performance Computing: https://cran.r-project.org/view=HighPerformanceComputing

nanonext website: https://shikokuchuo.net/nanonext/ nanonext on CRAN: https://cran.r-project.org/package=nanonext

NNG website: https://nng.nanomsg.org/

« Back to ToC

–

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.