chunked-class: Chunked Vectors, Arrays, and Lists

In kuwisdelu/matter: Out-of-core statistical computing and signal processing

Chunked Vectors, Arrays, and Lists

Description

The chunked class implements chunked wrappers for vectors, arrays, and lists for parallel iteration.

Usage

## Instance creation
chunked_vec(x, nchunks = NA, chunksize = NA,
    verbose = FALSE, permute = FALSE, depends = NULL, drop = FALSE)

chunked_mat(x, margin, nchunks = NA, chunksize = NA,
    verbose = FALSE, permute = FALSE, depends = NULL, drop = FALSE)

chunked_list(..., nchunks = NA, chunksize = NA,
    verbose = FALSE, permute = FALSE, depends = NULL, drop = FALSE)

## Additional methods documented below

Arguments

x, ...	The data to be chunked. If multiple objects are passed via ..., then they are all recycled to the same length.
nchunks	The number of chunks to use. If NA (the default), this is taken from getOption("matter.default.nchunks"). For IO-bound operations, using fewer chunks will often be faster, but use more memory. If both nchunks and chunksize are specified, then nchunks takes priority.
chunksize	The approximate chunk size in bytes. If NA (the default), this is taken from getOption("matter.default.chunksize"). For IO-bound operations, using larger chunks will often be faster, but use more memory. If both nchunks and chunksize are specified, then nchunks takes priority.
verbose	Should messages be printed whenever a chunk is extracted?
permute	Should the order of items be randomized? Alternatively, an integer vector or a list of integer vectors can be specified. If an integer vector is provided, then x is chunked in the exact order of the provided indices. If a list of indices is provided, then these are taken as strata (i.e., subpopulations). Each stratum will be chunked separately and then merged (without randomization), so that each chunk will contain examples from every stratum.
depends	A list with length equal to the extent of X. Each element of depends should give a vector of indices which correspond to other elements of X on which each computation depends. These elements are passed to FUN. For time efficiency, no attempt is made to verify these indices are valid.
margin	Which array margin should be chunked.
drop	The value passed to drop when subsetting the chunks.

Value

An object derived from class chunked.

Slots

data:

The data.

index:

The chunk indices.

verbose:

Print messages on chunk extraction?

drop:

The value passed to drop when subsetting the chunks.

margin:

The array margin for the chunks.

Creating Objects

chunked_vec, chunked_mat and chunked_list instances can be created through chunked_vec(), chunked_mat(), and chunked_list(), respectively.

Methods

Standard generic methods:

length(x):

Get number of chunks.

lengths(x):

Get chunk sizes for each chunk.

x[i, ...]:

Get chunks.

x[[i]]:

Get a single chunk.

Author(s)

Kylie A. Bemis

See Also

chunkApply

Examples

x <- matrix(runif(200), nrow=10, ncol=20)

y <- chunked_mat(x, margin=2, nchunks=5)
print(y)

kuwisdelu/matter documentation built on Dec. 8, 2024, 8:09 p.m.