bdReduce_hdf5_dataset: Reduce Multiple HDF5 Datasets

In BigDataStatMeth: Title: Scalable Statistical Computing with HDF5-Backed Matrices

Reduce Multiple HDF5 Datasets

Description

Reduces multiple datasets within an HDF5 group using arithmetic operations (addition or subtraction).

Usage

bdReduce_hdf5_dataset(
  filename,
  group,
  reducefunction,
  outgroup = NULL,
  outdataset = NULL,
  overwrite = FALSE,
  remove = FALSE
)

Arguments

filename	Character string. Path to the HDF5 file.
group	Character string. Path to the group containing datasets.
reducefunction	Character. Operation to apply, either "+" or "-".
outgroup	Character string (optional). Output group path. If NULL, uses input group.
outdataset	Character string (optional). Output dataset name. If NULL, uses input group name.
overwrite	Logical (optional). Whether to overwrite existing dataset. Default is FALSE.
remove	Logical (optional). Whether to remove source datasets after reduction. Default is FALSE.

Details

This function provides efficient dataset reduction capabilities with:

• Operation options:

  • Addition of datasets

  • Subtraction of datasets

• Output options:

  • Custom output location

  • Configurable dataset name

  • Overwrite protection

• Implementation features:

  • Memory-efficient processing

  • Safe file operations

  • Optional source cleanup

  • Comprehensive error handling

The function processes datasets efficiently while maintaining data integrity.

Value

List with components. If an error occurs, all string values are returned as empty strings (""):

fn

Character string with the HDF5 filename

ds

Character string with the full dataset path to the reduced dataset (group/dataset)

func

Character string with the reduction function applied

References

• The HDF Group. (2000-2010). HDF5 User's Guide.

Examples

fn <- tempfile(fileext = ".h5")
hdf5_create_matrix(fn, "data/matrix1", data = matrix(1:100, 10, 10))
hdf5_create_matrix(fn, "data/matrix2", data = matrix(101:200, 10, 10))
hdf5_create_matrix(fn, "data/matrix3", data = matrix(201:300, 10, 10))

bdReduce_hdf5_dataset(
  filename   = fn,
  group      = "data",
  reducefunction = "+",
  outgroup   = "results",
  outdataset = "sum_matrix",
  overwrite  = TRUE
)
hdf5_close_all()
unlink(fn)

BigDataStatMeth documentation built on May 15, 2026, 1:07 a.m.