mvl_open: Open an MVL file

In RMVL: Mappable Vector Library for Handling Large Datasets

Open an MVL file

Description

Open an MVL format file for reading and/or writing.

Usage

mvl_open(filename, append = FALSE, create = FALSE)

Arguments

filename	path to file.
append	specify TRUE when you intend to write data into the file
create	when TRUE create file if it did not exist

Details

MVL stands for "Mapped vector library" and is a file format designed for efficient memory mapped access. An MVL file can be much larger than physical memory of the machine.

mvl_open returns a handle that can be used to access MVL files. Files opened read-only are memory mapped and do not use a file descriptor, and thus are not subject to limits on the number of open files. Files opened for writing data do use a file descriptor. Once opened for read access the data can be accessed using usual R semantics for lists, data.frames and arrays.

Value

handle to opened MVL file

See Also

mvl_close, mvl_remap

Examples

## Not run: 
M1<-mvl_open("test1.mvl", append=TRUE, create=TRUE)
mvl_write_object(M1, data.frame(x=1:2, y=rnorm(2)), "test_frame")
mvl_close(M1)

M2<-mvl_open("test1.mvl")
print(names(M2))
print(M2["test_frame"])
mvl_close(M2)

M3<-mvl_open("test2.mvl", append=TRUE, create=TRUE)
L<-list()
df<-data.frame(x=1:1e6, y=rnorm(1e6), s=rep(c("a", "b"), 5e5))
L[["x"]]<-mvl_write_object(M3, df, drop.rownames=TRUE)
L[["description"]]<-"Example of large data frame"
mvl_write_object(M3, L, "test_object")
mvl_close(M3)

M4<-mvl_open("test2.mvl")
print(names(M4))
L<-M4["test_object"]
print(L)
print(L[["x"]][1:20,])
mvl_object_stats(L[["x"]])
# If you need to get the whole x, one can use mvl2R(L[["x"]])
mvl_close(M4)

## End(Not run)

RMVL documentation built on Nov. 2, 2023, 6:09 p.m.