CFUN: Collapsing functions for batch processing

In ff: Memory-Efficient Storage of Large Data on Disk and Fast Access Functions

Collapsing functions for batch processing

Description

These are used in aggregating the chunks resulting from batch processing. They are usually called via do.call

Usage

ccbind(...)
crbind(...)
cfun(..., FUN, FUNARGS = list())
cquantile(..., probs = seq(0, 1, 0.25), na.rm = FALSE, names = TRUE, type = 7)
csummary(..., na.rm = "ignored")
cmedian(..., na.rm = FALSE)
clength(..., na.rm = FALSE)
csum(..., na.rm = FALSE)
cmean(..., na.rm = FALSE)

Arguments

...	...
FUN	a aggregating function
FUNARGS	further arguments to the aggregating function
na.rm	TRUE to remove NAs
probs	see quantile
names	see quantile
type	see quantile

Details

CFUN	FUN	comment
ccbind	cbind	like cbind but respecting names
crbind	rbind	like rbind but respecting names
cfun		crbind the input chunks and then apply 'FUN' to each column
cquantile	quantile	crbind the input chunks and then apply 'quantile' to each column
csummary	summary	crbind the input chunks and then apply 'summary' to each column
cmedian	median	crbind the input chunks and then apply 'median' to each column
clength	length	crbind the input chunks and then determine the number of values in each column
csum	sum	crbind the input chunks and then determine the sum values in each column
cmean	mean	crbind the input chunks and then determine the (unweighted) mean in each column

In order to use CFUNs on the result of lapply or ffapply use do.call.

Value

depends on the CFUN used

ff options

xx TODO: extend this for weighted means, weighted median etc.,
google "Re: [R] Weighted median"

Note

Currently - for command line convenience - we map the elements of a single list argument to ..., but this may change in the future.

Author(s)

Jens Oehlschlägel

See Also

ffapply, do.call, na.count

Examples

   X <- lapply(split(rnorm(1000), 1:10), summary)
   do.call("crbind", X)
   do.call("csummary", X)
   do.call("cmean", X)
   do.call("cfun", c(X, list(FUN=mean, FUNARGS=list(na.rm=TRUE))))
   rm(X)

ff documentation built on Sept. 30, 2024, 9:38 a.m.