mcsaveRDS: Multi-threaded saveRDS

In traversc/trqwe: Performance oriented statistical metrics and utility functions.

Description

Uses the pigz utility to improve saving large R objects. This is compatible with saveRDS and readRDS functions. Requires pigz (sudo apt-get install pigz on Ubuntu).

Usage

mcsaveRDS(object, file, mc.cores = min(parallel::detectCores(), 4))

Arguments

object	An r object to save.
file	The filename to save to.
mc.cores	How many cores to use in pigz. The program does not seem to benefit after more than about 4 cores.

See Also

http://stackoverflow.com/questions/28927750/

Examples

x <- sample(1e4, 1e7, replace=T)
y <- sample(1e4, 1e7, replace=T)
microbenchmark(mcsaveRDS(x, file="temp.Rds"), saveRDS(y, file="temp2.Rds")

Unit: seconds
                            expr      min       lq     mean   median       uq
 mcsaveRDS(x, file = "temp.Rds") 1.908310 1.908310 1.908310 1.908310 1.908310
  saveRDS(y, file = "temp2.Rds") 6.271499 6.271499 6.271499 6.271499 6.271499

traversc/trqwe documentation built on Dec. 4, 2020, 4:21 a.m.