LazyFST: R6 Class to Load 'fst' Files

In dipterix/raveutils: Utility Toolbox for RAVE package

Description

provides hybrid data structure for 'fst' file

Methods

Public methods

• LazyFST$open()

• LazyFST$close()

• LazyFST$save()

• LazyFST$new()

• LazyFST$get_dims()

• LazyFST$subset()

---

Method open()

to be compatible with LazyH5

Usage

LazyFST$open(...)

Arguments

...

ignored

Returns

none

---

Method close()

close the connection

Usage

LazyFST$close(..., .remove_file = FALSE)

Arguments

...

ignored

.remove_file

whether to remove the file when garbage collected

Returns

none

---

Method save()

to be compatible with LazyH5

Usage

LazyFST$save(...)

Arguments

...

ignored

Returns

none

---

Method new()

constructor

Usage

LazyFST$new(file_path, transpose = FALSE, dims = NULL, ...)

Arguments

file_path

where the data is stored

transpose

whether to load data transposed

dims

data dimension, only support 1 or 2 dimensions

...

ignored

---

Method get_dims()

get data dimension

Usage

LazyFST$get_dims(...)

Arguments

...

ignored

Returns

vector, dimensions

---

Method subset()

subset data

Usage

LazyFST$subset(i = NULL, j = NULL, ..., drop = TRUE)

Arguments

i, j, ...

index along each dimension

drop

whether to apply drop the subset

Returns

subset of data

Author(s)

Zhengjia Wang

Examples

# Data to save, total 8 MB
x <- matrix(rnorm(1000000), ncol = 100)

# Save to local disk
f <- tempfile()
fst::write_fst(as.data.frame(x), path = f, compress = 100)

# Load via LazyFST
dat <- LazyFST$new(file_path = f, dims = c(10000, 100))

pryr::object_size(dat)
#> 236 kB

# Check whether the data is identical
range(dat[] - x)

# The reading of column is very fast
system.time(dat[,100])

# Reading rows might be slow
system.time(dat[1,])

dipterix/raveutils documentation built on July 6, 2020, 12:24 a.m.