prcomp.HDF5Matrix: Principal Component Analysis of an HDF5Matrix

In BigDataStatMeth: Scalable Statistical Computing with HDF5-Backed Matrices

Principal Component Analysis of an HDF5Matrix

Description

Block-wise PCA entirely on disk, equivalent to prcomp(). Implements the same interface as stats::prcomp() but operates on data stored in an HDF5 file without loading it into RAM.

Usage

## S3 method for class 'HDF5Matrix'
prcomp(
  x,
  retx = TRUE,
  center = TRUE,
  scale. = FALSE,
  tol = NULL,
  rank. = NULL,
  ncomponents = 0L,
  k = 2L,
  q = 1L,
  method = "auto",
  rankthreshold = 0,
  svdgroup = "SVD/",
  overwrite = FALSE,
  threads = -1L,
  ...
)

Arguments

x	An HDF5Matrix object.
retx	Logical. If TRUE (default) return the individual coordinates (x slot). If FALSE the x slot is NULL in the returned object.
center	Logical. Subtract column means before PCA (default TRUE).
scale.	Logical. Divide by column SDs before PCA (default FALSE).
tol	Ignored (present for interface compatibility with prcomp()).
rank.	Integer. Number of principal components to compute. When supplied (non-NULL), takes precedence over ncomponents. Present for compatibility with stats::prcomp(); unlike the base R method, it is not ignored here.
ncomponents	Integer. Number of PCs to compute (0 = all, default).
k	Number of local SVDs per incremental level (default 2).
q	Number of incremental levels (default 1).
method	Computation method: "auto" (default), "blocks", or "full".
rankthreshold	Numeric in [0, 0.1]. Rank approximation threshold.
svdgroup	HDF5 group for intermediate SVD storage (default "SVD/").
overwrite	Logical. Recompute even if PCA results exist (default FALSE).
threads	Integer. OpenMP threads (-1 = auto-detect). The effective number may be lower than requested: the system ceiling (OMP_NUM_THREADS, OMP_THREAD_LIMIT and, by default, half of the detected cores) is applied silently by the OpenMP runtime. Requesting more than that is not an error, but it now emits a warning stating how many threads can actually be used.
...	Ignored (S3 compatibility).

Details

Constant columns and scale. = TRUE. As in stats::prcomp(), a column with zero variance cannot be rescaled to unit variance and the call stops with an error reporting how many columns are affected and a few of their positions. Previously the division by zero propagated silently and the whole decomposition degenerated to zeros. Either leave scale. = FALSE (the default) or remove the constant columns first.

Value

An object of class c("HDF5PCA", "list") with elements:

sdev

Numeric vector. Standard deviations of the PCs.

rotation

HDF5Matrix. Variable loadings (rotation matrix).

x

HDF5Matrix or NULL. Individual coordinates.

center

Logical. Whether columns were centered.

scale

Logical. Whether columns were scaled.

cumvar

Numeric vector. Cumulative variance explained (percent).

lambda

Numeric vector. Eigenvalues.

var.cos2

HDF5Matrix. Squared cosines for variables.

ind.cos2

HDF5Matrix. Squared cosines for individuals.

ind.contrib

HDF5Matrix. Contributions of individuals to PCs.

file

Character. Path to the HDF5 file with all results.

The object also carries the attributes method, exact, elements, auto_threshold, blocking and rank, recording which decomposition path was actually taken; print() shows them.

Exact versus approximate decomposition

PCA runs on the same machinery as svd.HDF5Matrix and inherits both of its silent approximations. Read that help page before interpreting a large PCA. In particular, supplying ncomponents or rank. below min(dim(x)) switches on per-block rank truncation, which is the larger error source and can perturb even the first principal component by a few percent; asking for a generously oversampled number of components and discarding the extra ones is much more accurate than asking for exactly the few you want. Check attr(pca, "exact") and attr(pca, "truncated"); print() shows both.

Examples

tmp <- tempfile(fileext = ".h5")
X   <- hdf5_create_matrix(tmp, "data/M", data = matrix(rnorm(1000), 100, 10))
pca <- prcomp(X, center = TRUE, scale. = FALSE)
cat("Variance explained (PC1-3):", pca$cumvar[1:3], "\n")
dim(pca$rotation)   # 10 x nPC
dim(pca$x)          # 100 x nPC

# rank. takes precedence over ncomponents when both are supplied.
# NOTE: this reuses the same output location as the call above, so it
# must come after pca's results have already been read/used -- once
# overwrite = TRUE recreates the output datasets, any earlier
# HDF5Matrix handle pointing at that location (here, pca$rotation and
# pca$x) becomes invalid by design (see HDF5Matrix lifecycle, Section
# 5.3.3): the package proactively clears stale external pointers when
# overwriting to convert what would otherwise be a memory-unsafe crash
# into a clean R-level error.

pca5 <- prcomp(X, rank. = 5, ncomponents = 10, overwrite = TRUE)
dim(pca5$rotation)   # 10 x 5 -- rank. (5) was used, not ncomponents (10)

hdf5_close_all()
unlink(tmp)

BigDataStatMeth documentation built on Sept. 15, 2026, 1:08 a.m.