Description Usage Arguments Value Examples
Parallelizes the writing of separate CSV files (still sequential reading) in order to store them in fst
format (also, overwrites fst::threads_fst
. Requires data.table
and fst
packages.
1 2 3 4 |
file |
Type: vector of characters. Path to all files to read. |
compress |
Type: numeric. Compression rate to use. Defaults to |
progress_bar |
Type: logical. Whether to print a progress bar. Defaults to |
clean_mem |
Type: logical. Whether the force garbage collection at the end of each file read in order to reclaim RAM. Defaults to |
cl |
Type: cluster or integer. A parallel cluster for parallelized calls. Used only when |
max_threads |
Type: numeric. The maximum number of threads allowed to adapt |
wkdir |
Type: character. The working directory, when using a cluster. Defaults to |
... |
Other arguments to pass to |
The element or the list of fst
file names.
1 2 3 4 5 6 7 8 9 10 11 | ## Not run:
# Cannot pass CRAN checks. Disabled.
# Do it on your own files!
library(fst) # devtools::install_github("fstPackage/fst@e060e62")
library(data.table)
library(parallel)
parallel.csv(c("file_1.csv", "file_2.csv"), max_threads = 1, progress_bar = TRUE)
parallel.csv(paste0("file_", 1:100, ".csv"), max_threads = 1, progress_bar = TRUE, cl = 8)
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.