crew: A Distributed Worker Launcher Framework

In computationally demanding analysis projects, statisticians and data scientists asynchronously deploy long-running tasks to distributed systems, ranging from traditional clusters to cloud services. The 'NNG'-powered 'mirai' R package by Gao (2023) <doi:10.5281/zenodo.7912722> is a sleek and sophisticated scheduler that efficiently processes these intense workloads. The 'crew' package extends 'mirai' with a unifying interface for third-party worker launchers. Inspiration also comes from packages. 'future' by Bengtsson (2021) <doi:10.32614/RJ-2021-048>, 'rrq' by FitzJohn and Ashton (2023) <https://github.com/mrc-ide/rrq>, 'clustermq' by Schubert (2019) <doi:10.1093/bioinformatics/btz284>), and 'batchtools' by Lang, Bischel, and Surmann (2017) <doi:10.21105/joss.00135>.

Package details

AuthorWilliam Michael Landau [aut, cre] (<https://orcid.org/0000-0003-1878-3253>), Daniel Woodie [ctb], Eli Lilly and Company [cph, fnd]
MaintainerWilliam Michael Landau <will.landau.oss@gmail.com>
LicenseMIT + file LICENSE
Version1.1.0
URL https://wlandau.github.io/crew/ https://github.com/wlandau/crew
Package repositoryView on CRAN
Installation Install the latest version of this package by entering the following in R:
install.packages("crew")

Try the crew package in your browser

Any scripts or data that you put into this service are public.

crew documentation built on April 11, 2025, 6:17 p.m.