LazyFST: R6 Class to Load 'fst' Files

In raveio: File-System Toolbox for RAVE Project

R6 Class to Load 'fst' Files

Description

provides hybrid data structure for 'fst' file

Value

none

none

none

vector, dimensions

subset of data

Methods

Public methods

• LazyFST$open()

• LazyFST$close()

• LazyFST$save()

• LazyFST$new()

• LazyFST$get_dims()

• LazyFST$subset()

---

Method open()

to be compatible with LazyH5

Usage

LazyFST$open(...)

Arguments

...

ignored

---

Method close()

close the connection

Usage

LazyFST$close(..., .remove_file = FALSE)

Arguments

...

ignored

.remove_file

whether to remove the file when garbage collected

---

Method save()

to be compatible with LazyH5

Usage

LazyFST$save(...)

Arguments

...

ignored

---

Method new()

constructor

Usage

LazyFST$new(file_path, transpose = FALSE, dims = NULL, ...)

Arguments

file_path

where the data is stored

transpose

whether to load data transposed

dims

data dimension, only support 1 or 2 dimensions

...

ignored

---

Method get_dims()

get data dimension

Usage

LazyFST$get_dims(...)

Arguments

...

ignored

---

Method subset()

subset data

Usage

LazyFST$subset(i = NULL, j = NULL, ..., drop = TRUE)

Arguments

i, j, ...

index along each dimension

drop

whether to apply drop the subset

Author(s)

Zhengjia Wang

Examples

if(!is_on_cran()){

# Data to save, total 8 MB
x <- matrix(rnorm(1000000), ncol = 100)

# Save to local disk
f <- tempfile()
fst::write_fst(as.data.frame(x), path = f)

# Load via LazyFST
dat <- LazyFST$new(file_path = f, dims = c(10000, 100))

# dat < 1 MB

# Check whether the data is identical
range(dat[] - x)

# The reading of column is very fast
system.time(dat[,100])

# Reading rows might be slow
system.time(dat[1,])

}

raveio documentation built on July 26, 2023, 5:29 p.m.