mvl_start_write_vector: Piecewise output of very long numeric and integer vectors

In RMVL: Mappable Vector Library for Handling Large Datasets

Piecewise output of very long numeric and integer vectors

Description

While mvl_fused_write_objects can be used to create very large vectors and data frames of arbitrary type, it requires piecewise data to be written first into an MVL file. Functions mvl_start_write_vector() and mvl_rewrite_vector() provide a way to create very long vectors in one pass. Only numeric and integer vectors are supported.

Usage

mvl_start_write_vector(MVLHANDLE, x, expected.length = NULL, name = NULL)

mvl_rewrite_vector(obj, offset, x)

Arguments

MVLHANDLE	handle to opened MVL file as generated by mvl_open()
x	an integer or numeric vector
expected.length	the length of vector to create. Use double to pass large values
name	if specified add a named entry to MVL file directory
obj	an MVL vector object to modify
offset	the offset into MVL vector (starting with 1) to write x

Details

One convenient use is to compute f(x,y,z,...) with very long vector arguments by iterating over indices. The iteration can be done using fixed blocks of indices, or by using groups of indices computed with other MVL functions.

It is generally recommended to call mvl_rewrite_vector() with large blocks to improve I/O performance and reduce number of writes to underlying media.

See Also

mvl_fused_write_objects

Examples

## Not run: 
Mtmp<-mvl_open("tmp_a.mvl", append=TRUE, create=TRUE)
offset<-mvl_start_write_object(Mtmp, runif(10), expected.length=1000, "vec1")
Mtmp<-mvl_remap(Mtmp)
mvl_rewrite_vector(Mtmp[offset], 50, rnorm(20))

## End(Not run)

RMVL documentation built on Feb. 23, 2025, 3:01 a.m.