precision: Measures of Precision and Shrinkage for Ridge Regression

In friendly/genridge: Generalized Ridge Trace Plots for Ridge Regression

Measures of Precision and Shrinkage for Ridge Regression

Description

Calculates measures of precision based on the size of the estimated covariance matrices of the parameters and shrinkage of the parameters in a ridge regression model. function does. ~~

Three measures of (inverse) precision based on the “size” of the covariance matrix of the parameters are calculated. Let V_k be the covariance matrix for a given ridge constant, and let \lambda_i , i= 1, \dots p be its eigenvalues

1. \log | V_k | = \log \prod \lambda or |V_k|^{1/p} =(\prod \lambda)^{1/p} measures the linearized volume of the covariance ellipsoid and corresponds conceptually to Wilks' Lambda criterion

2. trace( V_k ) = \sum \lambda corresponds conceptually to Pillai's trace criterion

3. \lambda_1 = max (\lambda) corresponds to Roy's largest root criterion.

Usage

precision(object, det.fun, normalize, ...)

Arguments

object	An object of class ridge or lm
det.fun	Function to be applied to the determinants of the covariance matrices, one of c("log","root").
normalize	If TRUE the length of the coefficient vector is normalized to a maximum of 1.0.
...	Other arguments (currently unused)

Value

A data.frame with the following columns

lambda	The ridge constant
df	The equivalent effective degrees of freedom
det	The det.fun function of the determinant of the covariance matrix
trace	The trace of the covariance matrix
max.eig	Maximum eigen value of the covariance matrix
norm.beta	The root mean square of the estimated coefficients, possibly normalized

Note

Models fit by lm and ridge use a different scaling for the predictors, so the results of precision for an lm model will not correspond to those for ridge with ridge constant = 0.

Author(s)

Michael Friendly

See Also

ridge,

Examples

longley.y <- longley[, "Employed"]
longley.X <- data.matrix(longley[, c(2:6,1)])

lambda <- c(0, 0.005, 0.01, 0.02, 0.04, 0.08)
lridge <- ridge(longley.y, longley.X, lambda=lambda)
clr <- c("black", rainbow(length(lambda)-1, start=.6, end=.1))
coef(lridge)

(pdat <- precision(lridge))
# plot log |Var(b)| vs. length(beta)
with(pdat, {
	plot(norm.beta, det, type="b", 
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: log |Var(b)|')
	text(norm.beta, det, lambda, cex=1.25, pos=c(rep(2,length(lambda)-1),4))
	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

# plot trace[Var(b)] vs. length(beta)
with(pdat, {
	plot(norm.beta, trace, type="b",
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: trace [Var(b)]')
	text(norm.beta, trace, lambda, cex=1.25, pos=c(2, rep(4,length(lambda)-1)))
#	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

friendly/genridge documentation built on March 4, 2024, 3:22 a.m.