hdf5_apply: Apply a mathematical operation to multiple HDF5 datasets

In BigDataStatMeth: Title: Scalable Statistical Computing with HDF5-Backed Matrices

Apply a mathematical operation to multiple HDF5 datasets

Description

Applies one of several supported operations to a list of datasets stored in an HDF5 group. Delegates to bdapply_Function_hdf5().

Usage

hdf5_apply(
  filename,
  group,
  datasets,
  func,
  outgroup,
  b_group = NULL,
  b_datasets = NULL,
  overwrite = FALSE,
  transp_dataset = FALSE,
  transp_bdataset = FALSE,
  fullMatrix = FALSE,
  byrows = FALSE,
  threads = NULL
)

Arguments

filename	Path to the HDF5 file.
group	Group path containing datasets.
datasets	Character vector of dataset names to process.
func	Character. Operation to apply (see Details).
outgroup	Character. Output group path for results.
b_group	Character or NULL. Group of B datasets.
b_datasets	Character vector or NULL. Names of B datasets.
overwrite	Logical. Overwrite existing output datasets.
transp_dataset	Logical. Transpose A datasets before operation.
transp_bdataset	Logical. Transpose B datasets before operation.
fullMatrix	Logical. Return full matrix (not triangular).
byrows	Logical. Apply by rows (for normalize/sdmean).
threads	Integer or NULL. OpenMP threads.

Details

Supported values for func:

"CrossProd"

Compute A^T A for each dataset.

"tCrossProd"

Compute A A^T for each dataset.

"CrossProd_double"

Double-precision cross-product.

"tCrossProd_double"

Double-precision transposed cross-product.

"blockmult"

Block-wise A \times B (requires b_datasets).

"QR"

QR decomposition for each dataset.

"invChol"

Inverse via Cholesky for each dataset.

"solve"

Solve AX = B (requires b_datasets).

"normalize"

Column-wise normalization.

"sdmean"

Compute SD and mean.

"descChol"

Cholesky decomposition.

Value

Invisibly NULL. Results written to outgroup. Open them with hdf5_matrix().

See Also

hdf5_reduce, crossprod.HDF5Matrix, qr.HDF5Matrix

Examples

tmp <- tempfile(fileext = ".h5")
bdCreate_hdf5_matrix(tmp, matrix(rnorm(20), 4, 5), "inp", "A",
                     overwriteFile = FALSE)
bdCreate_hdf5_matrix(tmp, matrix(rnorm(20), 4, 5), "inp", "B",
                     overwriteFile = FALSE)
hdf5_apply(tmp, group = "inp", datasets = c("A", "B"),
           func = "CrossProd", outgroup = "out")
res_A <- hdf5_matrix(tmp, "out/A")
dim(res_A)   # 5 x 5
close(res_A)
unlink(tmp)

BigDataStatMeth documentation built on May 15, 2026, 1:07 a.m.