spca: Sparse principal component analysis

View source: R/spca.R

spcaR Documentation

Sparse principal component analysis

Description

Fit interpretable component scores as sparse linear combinations of the input variables. Variable projection alternates an exact orthogonal reconstruction update with a proximal step for the loading coefficients.

Usage

spca(
  X,
  k = NULL,
  alpha = 1e-04,
  beta = 1e-04,
  center = TRUE,
  scale = FALSE,
  max_iter = 1000,
  tol = 1e-05,
  verbose = TRUE,
  penalty = c("elasticnet", "l0"),
  B_init = NULL
)

Arguments

X

A real numeric matrix, data frame with numeric columns, or real Matrix object. Rows are observations and columns are variables. Rows containing missing values are omitted with a warning; at least two complete observations and one variable are required. Logical inputs are treated as zero/one. Infinite and complex values are not supported. spca converts Matrix input to a dense matrix; see rspca for computation that preserves sparse input storage.

k

Positive integer number of components. NULL uses min(nrow(X), ncol(X)) after incomplete rows are omitted. Larger values are reduced to that limit. A component can have zero scores when the data are rank deficient or its loadings are all zero.

alpha

Finite nonnegative sparsity penalty (default 1e-4), relative to the squared largest singular value of the processed data. The effective penalty is alpha * d1^2; see Details. Increasing alpha encourages fewer nonzero coefficients but does not impose an exact feature count.

beta

Finite nonnegative ridge penalty, using the same relative scaling as alpha. The default is 1e-4. Ridge shrinkage can improve conditioning for strongly correlated variables.

center

Logical; subtract column means before fitting (default TRUE). The fitted means are stored for prediction.

scale

Logical; divide each processed column by its scale factor (default FALSE). Factors are square roots of column sums of squares divided by n-1, after any centering. This is the sample standard deviation for centered data and a root-mean-square quantity otherwise. Factors below 1e-8 are replaced by one.

max_iter

Positive integer limit on optimization iterations. Reaching the limit does not establish convergence.

tol

Finite nonnegative tolerance for relative objective change. This controls an optimization stopping rule, not a reconstruction-error bound or a statistical accuracy guarantee. An objective increase larger than a roundoff allowance stops with a warning and does not count as convergence; see Details.

verbose

Logical, or the equivalent numeric zero/one; print optimization progress when enabled.

penalty

Sparsity penalty: "elasticnet" (the default) uses an entrywise absolute-value penalty and soft thresholding; "l0" counts nonzero coefficients and uses hard thresholding. With penalty = "l0" and beta > 0, the combination of cardinality and ridge penalties is also called ZeroNet.

B_init

Optional real finite p by k matrix of initial loading coefficients. NULL initializes from leading right singular vectors. A previous fit's loadings can supply a warm start when variable order, component count, and preprocessing match. Initial loadings are not normalized.

Details

Let X denote the retained, centered and/or scaled data, with n observations and p variables. The default method seeks to minimize

\frac{1}{2}\|X-XBA^T\|_F^2 + \lambda_1\|B\|_1 + \frac{\lambda_2}{2}\|B\|_F^2

subject to A^T A=I. Here A and B are p by k; \|B\|_1 is the sum of absolute entries, and \|\cdot\|_F is the Frobenius norm. The effective coefficients are \lambda_1=\mathtt{alpha}\,d_1^2 and \lambda_2=\mathtt{beta}\,d_1^2, where d_1 is the largest singular value of the processed matrix. For penalty = "l0", replace \|B\|_1 with the number of nonzero entries of B. The same numeric alpha does not imply equivalent sparsity under the two penalties.

For fixed B, an SVD of X^T X B gives the best orthogonal reconstruction factor A. A gradient step followed by thresholding updates B. The joint problem is nonconvex, so the solution can depend on initialization. Small objective change is a convergence diagnostic, not proof of a global minimum or a bound on the gradient. With alpha = beta = 0 and the default SVD initialization, deterministic fitting recovers ordinary PCA with matching preprocessing, up to signs and bases within repeated singular subspaces. A custom B_init can remain in a different stationary subspace even with zero penalties.

Relative improvement divides the decrease in objective by the magnitude of the current objective, with a positive machine denominator at zero. The allowance for numerical stagnation is 64 times machine precision times the largest of the consecutive objective magnitudes and the squared data norm. Machine precision is .Machine$double.eps. Larger increases stop with a warning and converged = FALSE. For rspca, the squared data norm in this allowance is that of the sketch. The returned A was optimized immediately before the final B update (and the final S update for robust fits), so objective convergence does not imply that every block's first-order optimality condition holds to tol. A tighter tolerance can refine those conditions, at the cost of more iterations.

Scores are Z=XB; reconstruction in processed coordinates is ZA^T. The loading matrix B retains its penalized scale. It is generally neither normalized nor orthogonal. The reconstruction factor A is not a matrix inverse of B. To reconstruct in input coordinates, multiply columns of ZA^T by the stored scales and then add the stored means, omitting operations stored as FALSE.

Value

A list of class "spca" containing:

loadings

A dense p by k matrix B of fitted sparse coefficients. Zeros are stored in an ordinary R matrix.

transform

A dense p by k reconstruction factor A with orthonormal columns.

scores

A dense n by k matrix XB. Rows correspond to complete observations retained from the input; available input row names are preserved.

eigenvalues

Numeric vector of length k containing sequentially adjusted score energies divided by n-1; see the variance discussion. The field name is retained for compatibility.

sdev

Square roots of eigenvalues, not generally the marginal standard deviations of correlated score columns.

score_variance

Numeric vector of marginal score squared norms divided by n-1; sample variances when centering is enabled.

var

Total squared energy of the processed input divided by n-1, including directions omitted from the model.

center, scale

Numeric vectors of length p containing training means and scale factors, respectively, or FALSE when the corresponding operation is disabled. They describe the preprocessing performed by the function, in training variable order.

objective

Numeric vector with one penalized objective value per completed optimization iteration, or a single zero for zero-energy input.

iterations

Number of stored objective values.

converged

Logical indicator of objective-based convergence. This is not a global-optimality certificate.

improvement

Relative change used by the final convergence check.

alpha, beta

Effective penalties, scaled by the squared leading singular value.

penalty

The selected sparsity penalty.

feature_names

Training column names used for prediction, or NULL for unnamed input.

The summary method returns a numeric matrix with four rows: adjusted variance, its square root, proportion of total input energy, and cumulative proportion. The historical row labels and three-decimal rounding are retained. The print method displays the fitted statistics and loadings.

Two matrices with different jobs

Ordinary PCA uses orthogonal loading vectors to make component scores and reconstruct the observations. Sparse PCA separates those jobs. Each column of loadings is a recipe for one score: a zero coefficient means that the corresponding variable is not directly used in that recipe. Each row of scores contains the resulting k coordinates for one observation. The dense transform maps those coordinates back to the p measured variables.

This distinction lets a component use a short, interpretable list of variables while its reconstruction describes relationships with many variables. A variable omitted from loadings can still have a nonzero reconstructed value through transform. Variables can contribute to more than one component. Sparsity concerns individual coefficients, so the total number of selected variables can be smaller than the sum of the per-component nonzero counts.

The fitted scale of loadings is part of the model. Normalizing its columns afterward changes scores, prediction, and the penalized objective. Use the original fitted coefficients for calculations; make a separate normalized copy only if a particular display requires it. A coefficient's magnitude or sign is not a significance test or a causal effect.

Preparing data and choosing parameters

Decide first what an observation and a variable represent. For example, rows might be repeated measurements and columns might be sensors. Transposing the data changes which objects receive sparse coefficients. Supply only numeric measurements; identifiers and outcome labels should not be included accidentally as additional variables. Missing rows are omitted, not imputed. Keep their original row identifiers if scores must later be joined to other data.

Centering asks for variation around the training means. Use center = FALSE when the origin itself has a meaningful role in the analysis. Scaling makes variables comparable in standardized units; it is useful when measurement units differ, but it also gives small variations in an otherwise nearly constant variable more influence. With center = FALSE, scaling uses uncentered squared norms with divisor n-1, not centered standard deviations. The absolute 1e-8 scale cutoff is retained for compatibility, so constant or extremely small columns are not divided by a near-zero number.

A practical tuning workflow is:

  1. Choose k for the representation you need, using domain knowledge and, where appropriate, held-out performance. It is the number of score coordinates, not the number of selected variables. The default k = NULL requests the full available count; it does not select a useful rank automatically.

  2. Fit a weakly penalized model, then examine a range of alpha values with spca_path. Inspect both nonzero counts and what the selected variables mean. A grid such as c(0.0001, 0.001, 0.01, 0.1) is an exploration of scales, not a generally optimal set of penalties. If the application suggests a desired number of measured variables, spca_target searches for a fit with a nearby count.

  3. Use beta to control ridge shrinkage and conditioning, particularly with correlated or redundant measurements. Increasing it also reduces loading magnitudes, so it changes the fitted model. The small default is a starting point, not an automatic selection rule.

  4. Inspect converged, the objective history, and reconstruction quality. If the iteration limit was reached, a larger max_iter permits more work. A tighter tol refines objective stopping but does not repair an unsuitable rank or penalty.

Compare fits under the same preprocessing. Objectives at different penalties contain different costs and should not be ranked as if they were the same goodness-of-fit measure. Reconstruction error and validation performance answer complementary questions to sparsity. Hard thresholding can give abrupt support changes; the same numeric alpha should not be interpreted as equivalent to soft thresholding. Zero processed data return zero loadings and scores with a single zero objective value and converged = TRUE. Finite inputs can still exceed floating-point limits when squared. If the squared norm overflows, or underflows to zero for nonzero processed data, fitting stops with an error asking for rescaling. Bring measurements to a moderate scale before fitting. Automatic column scaling cannot resolve every extreme input because its own scale calculation uses squared values and retains the 1e-8 cutoff.

Sparse coefficients and sparse storage

A sparse loading matrix has many zero coefficients. A sparse input matrix is stored using a class from Matrix to avoid storing its many zero entries. Neither property implies the other. spca accepts sparse input but makes a dense working matrix for the deterministic factorization. Use rspca when retaining sparse input storage is important and a small randomized sketch is appropriate. Both methods return ordinary dense loading and score matrices. Forming tcrossprod(scores, transform) also allocates an n by p dense reconstruction; retaining the factors is often preferable until a reconstruction is actually needed.

Variance and loading interpretation

Sparse component scores can be correlated. The historical eigenvalues field contains sequentially adjusted score energies: project each score column onto the span of preceding columns, subtract that projection, and divide the squared residual norm by n-1. This is equivalent to squared diagonal entries in an unpivoted QR decomposition of the scores, with dependent columns assigned zero. Rank is determined using base R's default QR tolerance, so numerically dependent score columns can also receive zero adjusted energy. This tolerance is separate from the optimizer's tol. The adjustment follows Zou, Hastie and Tibshirani (2006), applied to the returned, unnormalized loading coefficients. It depends on component order and loading scale; values are not necessarily decreasing.

For centered data these are adjusted sample variances. They are not generally eigenvalues of a covariance matrix. score_variance contains the marginal squared score norms divided by n-1, without adjustment for overlapping components. If centering is disabled, both quantities are uncentered second moments with that denominator, rather than centered sample variances.

var is the total squared energy of the entire processed input divided by n-1; for centered data it is the exact sum of input sample variances. Summary proportions use this denominator, including variation in omitted directions. They need not sum to one. For zero total energy, variance proportions are mathematically undefined; the summary displays zero by convention.

Loadings are coefficients, not variable–component correlations. For complete training data, correlations can be computed directly as cor(as.matrix(X), fit$scores). Multiplying loadings by sdev does not generally produce these correlations for sparse components. A constant variable or an all-zero score has no defined correlation, so exclude such columns when calculating correlations.

Adjusted score energy is not generally the same as the reduction in reconstruction error from the fitted reconstruction factor. To assess reconstruction, evaluate the residual of tcrossprod(scores, transform) in the same processed units as the fit. Keep an ordinary PCA reference at the same rank and preprocessing when judging the price of interpretability.

Note

Version 1.0.0 corrects the variance statistics: older versions used singular values from an auxiliary Procrustes update, which were not the variances of the returned scores. Loading signs and bases within tied singular subspaces may differ between valid numerical implementations.

References

Erichson, N. B., Zheng, P., Manohar, K., Brunton, S. L., Kutz, J. N., and Aravkin, A. Y. (2020). Sparse Principal Component Analysis via Variable Projection. SIAM Journal on Applied Mathematics, 80(2), 977–1002. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1137/18M1211350")}.

Zou, H., Hastie, T., and Tibshirani, R. (2006). Sparse Principal Component Analysis. Journal of Computational and Graphical Statistics, 15(2), 265–286. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/106186006X113430")}.

See Also

rspca for randomized fitting, robspca for sparse additive errors, spca_path for a penalty path, predict.spca for new observations, prcomp for ordinary PCA.

Examples

X <- as.matrix(iris[, 1:4])
fit <- spca(X, k = 2, alpha = 0.01, beta = 0.001,
            scale = TRUE, verbose = FALSE)
colSums(fit$loadings != 0)
summary(fit)

# Reconstruct in original measurement coordinates.
reconstructed <- tcrossprod(fit$scores, fit$transform)
reconstructed <- sweep(reconstructed, 2, fit$scale, "*")
reconstructed <- sweep(reconstructed, 2, fit$center, "+")

# Measure reconstruction error in the units used by the objective.
processed <- scale(X, center = fit$center, scale = fit$scale)
residual <- processed - tcrossprod(fit$scores, fit$transform)
norm(residual, "F") / norm(processed, "F")

# Read a component as a list of original variables and coefficients.
setNames(fit$loadings[, 1], colnames(X))
colnames(X)[fit$loadings[, 1] != 0]
cor(X, fit$scores)

# Hard thresholding with a ridge penalty (ZeroNet).
hard <- spca(X, k = 2, alpha = 0.005, beta = 0.001,
             penalty = "l0", scale = TRUE, verbose = FALSE)
colSums(hard$loadings != 0)

# Ordinary PCA with matching preprocessing is the unpenalized case.
ordinary <- spca(X, k = 2, alpha = 0, beta = 0,
                 scale = TRUE, verbose = FALSE)
reference <- stats::prcomp(X, rank. = 2, scale. = TRUE)
cbind(ordinary$sdev, reference$sdev[1:2])

sparsepca documentation built on Nov. 6, 2026, 5:06 p.m.