| spca | R Documentation |
Fit interpretable component scores as sparse linear combinations of the input variables. Variable projection alternates an exact orthogonal reconstruction update with a proximal step for the loading coefficients.
spca(
X,
k = NULL,
alpha = 1e-04,
beta = 1e-04,
center = TRUE,
scale = FALSE,
max_iter = 1000,
tol = 1e-05,
verbose = TRUE,
penalty = c("elasticnet", "l0"),
B_init = NULL
)
X |
A real numeric matrix, data frame with numeric columns, or
real Matrix object. Rows are observations and columns are
variables. Rows containing missing values are omitted with a warning;
at least two complete observations and one variable are required.
Logical inputs are treated as zero/one. Infinite and complex values
are not supported. |
k |
Positive integer number of components. |
alpha |
Finite nonnegative sparsity penalty (default |
beta |
Finite nonnegative ridge penalty, using the same relative
scaling as |
center |
Logical; subtract column means before fitting (default
|
scale |
Logical; divide each processed column by its scale factor
(default |
max_iter |
Positive integer limit on optimization iterations. Reaching the limit does not establish convergence. |
tol |
Finite nonnegative tolerance for relative objective change. This controls an optimization stopping rule, not a reconstruction-error bound or a statistical accuracy guarantee. An objective increase larger than a roundoff allowance stops with a warning and does not count as convergence; see Details. |
verbose |
Logical, or the equivalent numeric zero/one; print optimization progress when enabled. |
penalty |
Sparsity penalty: |
B_init |
Optional real finite |
Let X denote the retained, centered and/or scaled data, with
n observations and p variables. The default method seeks
to minimize
\frac{1}{2}\|X-XBA^T\|_F^2 +
\lambda_1\|B\|_1 + \frac{\lambda_2}{2}\|B\|_F^2
subject to A^T A=I. Here A and B are p by
k; \|B\|_1 is the sum of absolute entries, and
\|\cdot\|_F is the Frobenius norm. The effective coefficients
are \lambda_1=\mathtt{alpha}\,d_1^2 and
\lambda_2=\mathtt{beta}\,d_1^2, where d_1 is the largest
singular value of the processed matrix. For penalty = "l0",
replace \|B\|_1 with the number of nonzero entries of B.
The same numeric alpha does not imply equivalent sparsity under
the two penalties.
For fixed B, an SVD of X^T X B gives the best orthogonal
reconstruction factor A. A gradient step followed by thresholding
updates B. The joint problem is nonconvex, so the solution can
depend on initialization. Small objective change is a convergence
diagnostic, not proof of a global minimum or a bound on the gradient.
With alpha = beta = 0 and the default SVD initialization,
deterministic fitting recovers ordinary PCA with matching preprocessing,
up to signs and bases within repeated singular subspaces. A custom
B_init can remain in a different stationary subspace even with
zero penalties.
Relative improvement divides the decrease in objective by the magnitude
of the current objective, with a positive machine denominator at zero.
The allowance for numerical stagnation is 64 times machine precision
times the largest of the consecutive objective magnitudes and the
squared data norm. Machine precision is .Machine$double.eps.
Larger increases stop with a warning and converged = FALSE.
For rspca, the squared data norm in this allowance is that of
the sketch. The returned A was optimized immediately before the
final B update (and the final S update for robust fits),
so objective convergence does not imply that every block's first-order
optimality condition holds to tol. A tighter tolerance can refine
those conditions, at the cost of more iterations.
Scores are Z=XB; reconstruction in processed coordinates is
ZA^T. The loading matrix B retains its penalized scale.
It is generally neither normalized nor orthogonal. The reconstruction
factor A is not a matrix inverse of B. To reconstruct in
input coordinates, multiply columns of ZA^T by the stored scales
and then add the stored means, omitting operations stored as
FALSE.
A list of class "spca" containing:
A dense p by k matrix B of fitted
sparse coefficients. Zeros are stored in an ordinary R matrix.
A dense p by k reconstruction factor
A with orthonormal columns.
A dense n by k matrix XB. Rows
correspond to complete observations retained from the input;
available input row names are preserved.
Numeric vector of length k containing
sequentially adjusted score energies divided by n-1; see
the variance discussion. The field name is retained for compatibility.
Square roots of eigenvalues, not generally the
marginal standard deviations of correlated score columns.
Numeric vector of marginal score squared norms
divided by n-1; sample variances when centering is enabled.
Total squared energy of the processed input divided by
n-1, including directions omitted from the model.
Numeric vectors of length p containing
training means and scale factors, respectively, or FALSE
when the corresponding operation is disabled. They describe the
preprocessing performed by the function, in training variable order.
Numeric vector with one penalized objective value per completed optimization iteration, or a single zero for zero-energy input.
Number of stored objective values.
Logical indicator of objective-based convergence. This is not a global-optimality certificate.
Relative change used by the final convergence check.
Effective penalties, scaled by the squared leading singular value.
The selected sparsity penalty.
Training column names used for prediction, or
NULL for unnamed input.
The summary method returns a numeric matrix with four rows:
adjusted variance, its square root, proportion of total input energy,
and cumulative proportion. The historical row labels and three-decimal
rounding are retained. The print method displays the fitted
statistics and loadings.
Ordinary PCA uses orthogonal loading vectors to make component scores
and reconstruct the observations. Sparse PCA separates those jobs.
Each column of loadings is a recipe for one score: a zero
coefficient means that the corresponding variable is not directly used
in that recipe. Each row of scores contains the resulting
k coordinates for one observation. The dense transform
maps those coordinates back to the p measured variables.
This distinction lets a component use a short, interpretable list of
variables while its reconstruction describes relationships with many
variables. A variable omitted from loadings can still have a
nonzero reconstructed value through transform. Variables can
contribute to more than one component. Sparsity concerns individual
coefficients, so the total number of selected variables can be smaller
than the sum of the per-component nonzero counts.
The fitted scale of loadings is part of the model. Normalizing
its columns afterward changes scores, prediction, and the penalized
objective. Use the original fitted coefficients for calculations;
make a separate normalized copy only if a particular display requires it.
A coefficient's magnitude or sign is not a significance test or a
causal effect.
Decide first what an observation and a variable represent. For example, rows might be repeated measurements and columns might be sensors. Transposing the data changes which objects receive sparse coefficients. Supply only numeric measurements; identifiers and outcome labels should not be included accidentally as additional variables. Missing rows are omitted, not imputed. Keep their original row identifiers if scores must later be joined to other data.
Centering asks for variation around the training means. Use
center = FALSE when the origin itself has a meaningful role in
the analysis. Scaling makes variables comparable in standardized units;
it is useful when measurement units differ, but it also gives small
variations in an otherwise nearly constant variable more influence.
With center = FALSE, scaling uses uncentered squared norms with
divisor n-1, not centered standard deviations. The absolute
1e-8 scale cutoff is retained for compatibility, so constant or
extremely small columns are not divided by a near-zero number.
A practical tuning workflow is:
Choose k for the representation you need, using domain
knowledge and, where appropriate, held-out performance. It is the
number of score coordinates, not the number of selected variables.
The default k = NULL requests the full available count;
it does not select a useful rank automatically.
Fit a weakly penalized model, then examine a range of
alpha values with spca_path. Inspect both
nonzero counts and what the selected variables mean. A grid such as
c(0.0001, 0.001, 0.01, 0.1) is an exploration of scales,
not a generally optimal set of penalties.
If the application suggests a desired number of measured variables,
spca_target searches for a fit with a nearby count.
Use beta to control ridge shrinkage and conditioning,
particularly with correlated or redundant measurements. Increasing
it also reduces loading magnitudes, so it changes the fitted model.
The small default is a starting point, not an automatic selection rule.
Inspect converged, the objective history, and
reconstruction quality. If the iteration limit was reached, a larger
max_iter permits more work. A tighter tol refines
objective stopping but does not repair an unsuitable rank or penalty.
Compare fits under the same preprocessing. Objectives at different
penalties contain different costs and should not be ranked as if they
were the same goodness-of-fit measure. Reconstruction error and
validation performance answer complementary questions to sparsity.
Hard thresholding can give abrupt support changes; the same numeric
alpha should not be interpreted as equivalent to soft thresholding.
Zero processed data return zero loadings and scores with a single zero
objective value and converged = TRUE.
Finite inputs can still exceed floating-point limits when squared.
If the squared norm overflows, or underflows to zero for nonzero
processed data, fitting stops with an error asking for rescaling.
Bring measurements to a moderate scale before fitting. Automatic column
scaling cannot resolve every extreme input because its own scale
calculation uses squared values and retains the 1e-8 cutoff.
A sparse loading matrix has many zero coefficients. A sparse input
matrix is stored using a class from Matrix to avoid storing its
many zero entries. Neither property implies the other.
spca accepts sparse input but makes a dense working matrix for
the deterministic factorization. Use rspca when retaining
sparse input storage is important and a small randomized sketch is
appropriate. Both methods return ordinary dense loading and score
matrices. Forming tcrossprod(scores, transform) also allocates
an n by p dense reconstruction; retaining the factors is
often preferable until a reconstruction is actually needed.
Sparse component scores can be correlated. The historical
eigenvalues field contains sequentially adjusted score energies:
project each score column onto the span of preceding columns, subtract
that projection, and divide the squared residual norm by n-1.
This is equivalent to squared diagonal entries in an unpivoted QR
decomposition of the scores, with dependent columns assigned zero.
Rank is determined using base R's default QR tolerance, so numerically
dependent score columns can also receive zero adjusted energy. This
tolerance is separate from the optimizer's tol.
The adjustment follows Zou, Hastie and Tibshirani (2006), applied to
the returned, unnormalized loading coefficients. It depends on component
order and loading scale; values are not necessarily decreasing.
For centered data these are adjusted sample variances. They are not
generally eigenvalues of a covariance matrix. score_variance
contains the marginal squared score norms divided by n-1,
without adjustment for overlapping components. If centering is disabled,
both quantities are uncentered second moments with that denominator,
rather than centered sample variances.
var is the total squared energy of the entire processed input
divided by n-1; for centered data it is the exact sum of input
sample variances. Summary proportions use this denominator, including
variation in omitted directions. They need not sum to one. For zero
total energy, variance proportions are mathematically undefined; the
summary displays zero by convention.
Loadings are coefficients, not variable–component correlations.
For complete training data, correlations can be computed directly as
cor(as.matrix(X), fit$scores). Multiplying loadings by
sdev does not generally produce these correlations for sparse
components. A constant variable or an all-zero score has no defined
correlation, so exclude such columns when calculating correlations.
Adjusted score energy is not generally the same as the reduction in
reconstruction error from the fitted reconstruction factor. To assess
reconstruction, evaluate the residual of
tcrossprod(scores, transform) in the same processed units as
the fit. Keep an ordinary PCA reference at the
same rank and preprocessing when judging the price of interpretability.
Version 1.0.0 corrects the variance statistics: older versions used singular values from an auxiliary Procrustes update, which were not the variances of the returned scores. Loading signs and bases within tied singular subspaces may differ between valid numerical implementations.
Erichson, N. B., Zheng, P., Manohar, K., Brunton, S. L., Kutz, J. N., and Aravkin, A. Y. (2020). Sparse Principal Component Analysis via Variable Projection. SIAM Journal on Applied Mathematics, 80(2), 977–1002. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1137/18M1211350")}.
Zou, H., Hastie, T., and Tibshirani, R. (2006). Sparse Principal Component Analysis. Journal of Computational and Graphical Statistics, 15(2), 265–286. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/106186006X113430")}.
rspca for randomized fitting,
robspca for sparse additive errors,
spca_path for a penalty path,
predict.spca for new observations,
prcomp
for ordinary PCA.
X <- as.matrix(iris[, 1:4])
fit <- spca(X, k = 2, alpha = 0.01, beta = 0.001,
scale = TRUE, verbose = FALSE)
colSums(fit$loadings != 0)
summary(fit)
# Reconstruct in original measurement coordinates.
reconstructed <- tcrossprod(fit$scores, fit$transform)
reconstructed <- sweep(reconstructed, 2, fit$scale, "*")
reconstructed <- sweep(reconstructed, 2, fit$center, "+")
# Measure reconstruction error in the units used by the objective.
processed <- scale(X, center = fit$center, scale = fit$scale)
residual <- processed - tcrossprod(fit$scores, fit$transform)
norm(residual, "F") / norm(processed, "F")
# Read a component as a list of original variables and coefficients.
setNames(fit$loadings[, 1], colnames(X))
colnames(X)[fit$loadings[, 1] != 0]
cor(X, fit$scores)
# Hard thresholding with a ridge penalty (ZeroNet).
hard <- spca(X, k = 2, alpha = 0.005, beta = 0.001,
penalty = "l0", scale = TRUE, verbose = FALSE)
colSums(hard$loadings != 0)
# Ordinary PCA with matching preprocessing is the unpenalized case.
ordinary <- spca(X, k = 2, alpha = 0, beta = 0,
scale = TRUE, verbose = FALSE)
reference <- stats::prcomp(X, rank. = 2, scale. = TRUE)
cbind(ordinary$sdev, reference$sdev[1:2])
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.