LazyH5: Lazy 'HDF5' file loader

In dipterix/raveutils: Utility Toolbox for RAVE package

Description

provides hybrid data structure for 'HDF5' file

Methods

Public methods

• LazyH5$finalize()

• LazyH5$print()

• LazyH5$new()

• LazyH5$save()

• LazyH5$open()

• LazyH5$close()

• LazyH5$subset()

• LazyH5$get_dims()

---

Method finalize()

garbage collection method

Usage

LazyH5$finalize()

Returns

none

---

Method print()

overrides print method

Usage

LazyH5$print()

Returns

self instance

---

Method new()

constructor

Usage

LazyH5$new(file_path, data_name, read_only = FALSE)

Arguments

file_path

where data is stored in 'HDF5' format

data_name

the data stored in the file

read_only

whether to open the file in read-only mode. It's highly recommended to set this to be true, otherwise the file connection is exclusive.

Returns

self instance

---

Method save()

save data to a 'HDF5' file

Usage

LazyH5$save(
  x,
  chunk = "auto",
  level = 7,
  replace = TRUE,
  new_file = FALSE,
  force = TRUE,
  ctype = NULL,
  size = NULL,
  ...
)

Arguments

x

vector, matrix, or array

chunk

chunk size, length should matches with data dimension

level

compress level, from 1 to 9

replace

if the data exists in the file, replace the file or not

new_file

remove the whole file if exists before writing?

force

if you open the file in read-only mode, then saving objects to the file will raise error. Use force=TRUE to force write data

ctype

data type, see mode, usually the data type of x. Try mode(x) or storage.mode(x) as hints.

size

deprecated, for compatibility issues

...

passed to self open() method

---

Method open()

open connection

Usage

LazyH5$open(new_dataset = FALSE, robj, ...)

Arguments

new_dataset

only used when the internal pointer is closed, or to write the data

robj

data array to save

...

passed to createDataSet

---

Method close()

close connection

Usage

LazyH5$close(all = TRUE)

Arguments

all

whether to close all connections associated to the data file. If true, then all connections, including access from other programs, will be closed

---

Method subset()

subset data

Usage

LazyH5$subset(..., drop = FALSE, stream = FALSE, envir = parent.frame())

Arguments

drop

whether to apply drop the subset

stream

whether to read partial data at a time

envir

if i,j,... are expressions, where should the expression be evaluated

i, j, ...

index along each dimension

Returns

subset of data

---

Method get_dims()

get data dimension

Usage

LazyH5$get_dims(stay_open = TRUE)

Arguments

stay_open

whether to leave the connection opened

Returns

dimension of the array

Author(s)

Zhengjia Wang

Examples

# Data to save
x <- array(rnorm(1000), c(10,10,10))

# Save to local disk
f <- tempfile()
save_h5(x, file = f, name = 'x', chunk = c(10,10,10), level = 0)

# Load via LazyFST
dat <- LazyH5$new(file_path = f, data_name = 'x', read_only = TRUE)

dat
#> Class: H5D
#> Dataset: /x
#> Filename: ...
#> Access type: H5F_ACC_RDONLY
#> Datatype: H5T_IEEE_F64LE
#> Space: Type=Simple     Dims=10 x 10 x 10     Maxdims=Inf x Inf x Inf
#> Chunk: 11 x 11 x 11

# Check whether the data is identical
range(dat - x)

# Read a slice of the data
system.time(dat[,10,])

dipterix/raveutils documentation built on July 6, 2020, 12:24 a.m.