Description Usage Arguments Details Value Note Author(s) References See Also Examples
This function fits the specified ordinary least squares or
parsimonious regression (plsr, pcr, ridge, and lars methods)
depending on the arguments provided, and returns estimates of
coefficients and (co)variances in a monomvn
friendly
format
1 2 3 4 
X 

y 
matrix of responses 
method 
describes the type of parsimonious
(or shrinkage) regression, or ordinary least squares.
From the pls package we have 
p 
when performing regressions, 
ncomp.max 
maximal number of (principal) components to consider
in a 
validation 
method for cross validation when applying
a parsimonious regression method. The default setting
of 
verb 
whether or not to print progress indicators. The default
( 
quiet 
causes 
All method
s (except "lsr"
) require a scheme for
estimating the amount of variability explained by increasing numbers
of nonzero coefficients (or principal components) in the model.
Towards this end, the pls and lars packages support
10fold cross validation (CV) or leaveoneout (LOO) CV estimates of
root mean squared error. See pls and lars for
more details. The regress
function uses CV in all cases
except when nrow(X) <= 10
, in which case CV fails and
LOO is used. Whenever nrow(X) <= 3
pcr
fails, so plsr
is used instead.
If quiet = FALSE
then a warning
is given whenever the first choice for a regression fails.
For pls methods, RMSEs
are calculated for a number of components in 1:ncomp.max
where
a NULL
value for ncomp.max
it is replaced with
ncomp.max < min(ncomp.max, ncol(y), nrow(X)1)
which is the max allowed by the pls package.
Simple heuristics are used to select a small number of components
(ncomp
for pls), or number of coefficients (for
lars) which explains a large amount of the variability (RMSE).
The lars methods use a “onestandard error rule” outlined
in Section 7.10, page 216 of HTF below. The
pls package does not currently support the calculation of
standard errors for CV estimates of RMSE, so a simple linear penalty
for increasing ncomp
is used instead. The ridge constant
(lambda) for lm.ridge
is set using the optimize
function on the GCV
output.
regress
returns a list
containing
the components listed below.
call 
a copy of the function call as used 
method 
a copy of the 
ncomp 
depends on the 
lambda 
if 
b 
matrix containing the estimated regression coefficients,
with 
S 
(biased corrected) maximum likelihood estimate of residual covariance matrix 
The CV in plsr and lars are random in nature, and so
can be dependent on the random seed. Use validation="LOO"
for
deterministic (but slower) result
Be warned that the lars implementation of
"forward.stagewise"
can sometimes get stuck in
(what seems like) an infinite loop.
This is not a bug in the regress
function;
the bug has been reported to the authors of lars
Robert B. Gramacy rbg@vt.edu
BjornHelge Mevik and Ron Wehrens (2007). The pls Package: Principal Component and Partial Least Squares Regression in R. Journal of Statistical Software 18(2)
Bradley Efron, Trevor Hastie, Ian Johnstone and Robert Tibshirani
(2003).
Least Angle Regression (with discussion).
Annals of Statistics 32(2); see also
http://wwwstat.stanford.edu/~hastie/Papers/LARS/LeastAngle_2002.pdf
http://bobby.gramacy.com/r_packages/monomvn
monomvn
, blasso
,
lars
in the lars library,
lm.ridge
in the MASS library,
plsr
and pcr
in the
pls library
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22  ## following the lars diabetes example
data(diabetes)
attach(diabetes)
## Ordinary Least Squares regression
reg.ols < regress(x, y)
## Lasso regression
reg.lasso < regress(x, y, method="lasso")
## partial least squares regression
reg.plsr < regress(x, y, method="plsr")
## ridge regression
reg.ridge < regress(x, y, method="ridge")
## compare the coefs
data.frame(ols=reg.ols$b, lasso=reg.lasso$b,
plsr=reg.plsr$b, ridge=reg.ridge$b)
## summarize the posterior distribution of lambda2 and s2
detach(diabetes)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.