precision: Measures of Precision and Shrinkage for Ridge Regression

In genridge: Generalized Ridge Trace Plots for Ridge Regression

Measures of Precision and Shrinkage for Ridge Regression

Description

The goal of precision is to allow you to study the relationship between shrinkage of ridge regression coefficients and their precision directly by calculating measures of each.

Three measures of (inverse) precision based on the “size” of the covariance matrix of the parameters are calculated. Let V_k \equiv \text{Var}(\mathbf{\beta}_k) be the covariance matrix for a given ridge constant, and let \lambda_i , i= 1, \dots p be its eigenvalues. Then the variance (= 1/precision) measures are:

1. "det": \log | V_k | = \log \prod \lambda (with det.fun = "log", the default) or |V_k|^{1/p} =(\prod \lambda)^{1/p} (with det.fun = "root") measures the linearized volume of the covariance ellipsoid and corresponds conceptually to Wilks' Lambda criterion

2. "trace": \text{trace}( V_k ) = \sum \lambda corresponds conceptually to Pillai's trace criterion

3. "max.eig": \lambda_1 = \max (\lambda) corresponds to Roy's largest root criterion.

Two measures of shrinkage are also calculated:

• norm.beta: the root mean square of the coefficient vector \lVert\mathbf{\beta}_k \rVert, normalized to a maximum of 1.0 if normalize == TRUE (the default).

• norm.diff: the root mean square of the difference from the OLS estimate \lVert \mathbf{\beta}_{\text{OLS}} - \mathbf{\beta}_k \rVert. This measure is inversely related to norm.beta

A plot method, plot.precision facilitates making graphs of these quantities.

Usage

precision(object, det.fun, normalize, ...)

Arguments

object	An object of class ridge or lm
det.fun	Function to be applied to the determinants of the covariance matrices, one of c("log","root").
normalize	If TRUE the length of the coefficient vector \mathbf{\beta}_k is normalized to a maximum of 1.0.
...	Other arguments (currently unused)

Value

An object of class c("precision", "data.frame") with the following columns:

lambda	The ridge constant
df	The equivalent effective degrees of freedom
det	The det.fun function of the determinant of the covariance matrix
trace	The trace of the covariance matrix
max.eig	Maximum eigen value of the covariance matrix
norm.beta	The root mean square of the estimated coefficients, possibly normalized
norm.diff	The root mean square of the difference between the OLS solution (lambda = 0) and ridge solutions

Note

Models fit by lm and ridge use a different scaling for the predictors, so the results of precision for an lm model will not correspond to those for ridge with ridge constant = 0.

Author(s)

Michael Friendly

See Also

ridge, plot.precision

Examples

longley.y <- longley[, "Employed"]
longley.X <- data.matrix(longley[, c(2:6,1)])

lambda <- c(0, 0.005, 0.01, 0.02, 0.04, 0.08)
lridge <- ridge(longley.y, longley.X, lambda=lambda)

# same, using formula interface
lridge <- ridge(Employed ~ GNP + Unemployed + Armed.Forces + Population + Year + GNP.deflator, 
		data=longley, lambda=lambda)

clr <- c("black", rainbow(length(lambda)-1, start=.6, end=.1))
coef(lridge)

(pdat <- precision(lridge))
# plot log |Var(b)| vs. length(beta)
with(pdat, {
	plot(norm.beta, det, type="b", 
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: log |Var(b)|')
	text(norm.beta, det, lambda, cex=1.25, pos=c(rep(2,length(lambda)-1),4))
	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

# plot trace[Var(b)] vs. length(beta)
with(pdat, {
	plot(norm.beta, trace, type="b",
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: trace [Var(b)]')
	text(norm.beta, trace, lambda, cex=1.25, pos=c(2, rep(4,length(lambda)-1)))
#	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

genridge documentation built on April 3, 2025, 11:07 p.m.