Description Usage Arguments Details Value Author(s) References See Also Examples
Fit a generalized linear model via penalized maximum likelihood. The regularization path is computed for the lasso or elasticnet penalty at a grid of values for the regularization parameter lambda. Can deal with all shapes of data, including very large sparse data matrices. Fits linear, logistic and multinomial, poisson, and Cox regression models.
1 2 3 4 5 6 7 8 9 | glmnet(x, y, family=c("gaussian","binomial","poisson","multinomial","cox","mgaussian"),
weights, offset=NULL, alpha = 1, nlambda = 100,
lambda.min.ratio = ifelse(nobs<nvars,0.01,0.0001), lambda=NULL,
standardize = TRUE, intercept=TRUE, thresh = 1e-07, dfmax = nvars + 1,
pmax = min(dfmax * 2+20, nvars), exclude, penalty.factor = rep(1, nvars),
lower.limits=-Inf, upper.limits=Inf, maxit=100000,
type.gaussian=ifelse(nvars<500,"covariance","naive"),
type.logistic=c("Newton","modified.Newton"),
standardize.response=FALSE, type.multinomial=c("ungrouped","grouped"))
|
x |
input matrix, of dimension nobs x nvars; each row is an
observation vector. Can be in sparse matrix format (inherit from class |
y |
response variable. Quantitative for |
family |
Response type (see above) |
weights |
observation weights. Can be total counts if responses are proportion matrices. Default is 1 for each observation |
offset |
A vector of length |
alpha |
The elasticnet mixing parameter, with 0≤α≤ 1. The penalty is defined as (1-α)/2||β||_2^2+α||β||_1.
|
nlambda |
The number of |
lambda.min.ratio |
Smallest value for |
lambda |
A user supplied |
standardize |
Logical flag for x variable standardization, prior to
fitting the model sequence. The coefficients are always returned on
the original scale. Default is |
intercept |
Should intercept(s) be fitted (default=TRUE) or set to zero (FALSE) |
thresh |
Convergence threshold for coordinate descent. Each inner
coordinate-descent loop continues until the maximum change in the
objective after any coefficient update is less than |
dfmax |
Limit the maximum number of variables in the
model. Useful for very large |
pmax |
Limit the maximum number of variables ever to be nonzero |
exclude |
Indices of variables to be excluded from the model. Default is none. Equivalent to an infinite penalty factor (next item). |
penalty.factor |
Separate penalty factors can be applied to each
coefficient. This is a number that multiplies |
lower.limits |
Vector of lower limits for each coefficient;
default |
upper.limits |
Vector of upper limits for each coefficient;
default |
maxit |
Maximum number of passes over the data for all lambda values; default is 10^5. |
type.gaussian |
Two algorithm types are supported for (only)
|
type.logistic |
If |
standardize.response |
This is for the |
type.multinomial |
If |
The sequence of models implied by lambda
is fit by coordinate
descent. For family="gaussian"
this is the lasso sequence if
alpha=1
, else it is the elasticnet sequence.
For the other families, this is a lasso or elasticnet regularization path
for fitting the generalized linear regression
paths, by maximizing the appropriate penalized log-likelihood (partial likelihood for the "cox" model). Sometimes the sequence is truncated before nlambda
values of lambda
have been used, because of instabilities in
the inverse link functions near a saturated fit. glmnet(...,family="binomial")
fits a traditional logistic regression model for the
log-odds. glmnet(...,family="multinomial")
fits a symmetric multinomial model, where
each class is represented by a linear model (on the log-scale). The
penalties take care of redundancies. A two-class "multinomial"
model
will produce the same fit as the corresponding "binomial"
model,
except the pair of coefficient matrices will be equal in magnitude and
opposite in sign, and half the "binomial"
values.
Note that the objective function for "gaussian"
is
1/2 RSS/nobs + λ*penalty,
and for the other models it is
-loglik/nobs + λ*penalty.
Note also that for
"gaussian"
, glmnet
standardizes y to have unit variance
before computing its lambda sequence (and then unstandardizes the
resulting coefficients); if you wish to reproduce/compare results with other
software, best to supply a standardized y. The coefficients for any predictor variables
with zero variance are set to zero for all values of lambda.
The latest two features in glmnet are the family="mgaussian"
family and the type.multinomial="grouped"
option for
multinomial fitting. The former allows a multi-response gaussian model
to be fit, using a "group -lasso" penalty on the coefficients for each
variable. Tying the responses together like this is called
"multi-task" learning in some domains. The grouped multinomial allows the same penalty for the
family="multinomial"
model, which is also multi-responsed. For
both of these the penalty on the coefficient vector for variable j is
(1-α)/2||β_j||_2^2+α||β_j||_2.
When
alpha=1
this is a group-lasso penalty, and otherwise it mixes
with quadratic just like elasticnet.
An object with S3 class "glmnet","*"
, where "*"
is
"elnet"
, "lognet"
,
"multnet"
, "fishnet"
(poisson), "coxnet"
or "mrelnet"
for the various types of models.
call |
the call that produced this object |
a0 |
Intercept sequence of length |
beta |
For |
lambda |
The actual sequence of |
dev.ratio |
The fraction of (null) deviance explained (for |
nulldev |
Null deviance (per observation). This is defined to be 2*(loglike_sat -loglike(Null)); The NULL model refers to the intercept model, except for the Cox, where it is the 0 model. |
df |
The number of nonzero coefficients for each value of
|
dfmat |
For |
dim |
dimension of coefficient matrix (ices) |
nobs |
number of observations |
npasses |
total passes over the data summed over all lambda values |
offset |
a logical variable indicating whether an offset was included in the model |
jerr |
error flag, for warnings and errors (largely for internal debugging). |
Jerome Friedman, Trevor Hastie, Noah Simon and Rob Tibshirani
Maintainer: Trevor Hastie hastie@stanford.edu
Friedman, J., Hastie, T. and Tibshirani, R. (2008)
Regularization Paths for Generalized Linear Models via Coordinate
Descent, http://www.stanford.edu/~hastie/Papers/glmnet.pdf
Journal of Statistical Software, Vol. 33(1), 1-22 Feb 2010
http://www.jstatsoft.org/v33/i01/
Simon, N., Friedman, J., Hastie, T., Tibshirani, R. (2011)
Regularization Paths for Cox's Proportional Hazards Model via
Coordinate Descent, Journal of Statistical Software, Vol. 39(5)
1-13
http://www.jstatsoft.org/v39/i05/
Tibshirani, Robert., Bien, J., Friedman, J.,Hastie, T.,Simon,
N.,Taylor, J. and Tibshirani, Ryan. (2012)
Strong Rules for Discarding Predictors in Lasso-type Problems,
JRSSB vol 74,
http://www-stat.stanford.edu/~tibs/ftp/strong.pdf
Stanford Statistics Technical Report
Glmnet Vignette http://www.stanford.edu/~hastie/glmnet/glmnet_alpha.html
print
, predict
, coef
and plot
methods, and the cv.glmnet
function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | # Gaussian
x=matrix(rnorm(100*20),100,20)
y=rnorm(100)
fit1=glmnet(x,y)
print(fit1)
coef(fit1,s=0.01) # extract coefficients at a single value of lambda
predict(fit1,newx=x[1:10,],s=c(0.01,0.005)) # make predictions
#multivariate gaussian
y=matrix(rnorm(100*3),100,3)
fit1m=glmnet(x,y,family="mgaussian")
plot(fit1m,type.coef="2norm")
#binomial
g2=sample(1:2,100,replace=TRUE)
fit2=glmnet(x,g2,family="binomial")
#multinomial
g4=sample(1:4,100,replace=TRUE)
fit3=glmnet(x,g4,family="multinomial")
fit3a=glmnet(x,g4,family="multinomial",type.multinomial="grouped")
#poisson
N=500; p=20
nzc=5
x=matrix(rnorm(N*p),N,p)
beta=rnorm(nzc)
f = x[,seq(nzc)]%*%beta
mu=exp(f)
y=rpois(N,mu)
fit=glmnet(x,y,family="poisson")
plot(fit)
pfit = predict(fit,x,s=0.001,type="response")
plot(pfit,y)
#Cox
set.seed(10101)
N=1000;p=30
nzc=p/3
x=matrix(rnorm(N*p),N,p)
beta=rnorm(nzc)
fx=x[,seq(nzc)]%*%beta/3
hx=exp(fx)
ty=rexp(N,hx)
tcens=rbinom(n=N,prob=.3,size=1)# censoring indicator
y=cbind(time=ty,status=1-tcens) # y=Surv(ty,1-tcens) with library(survival)
fit=glmnet(x,y,family="cox")
plot(fit)
# Sparse
n=10000;p=200
nzc=trunc(p/10)
x=matrix(rnorm(n*p),n,p)
iz=sample(1:(n*p),size=n*p*.85,replace=FALSE)
x[iz]=0
sx=Matrix(x,sparse=TRUE)
inherits(sx,"sparseMatrix")#confirm that it is sparse
beta=rnorm(nzc)
fx=x[,seq(nzc)]%*%beta
eps=rnorm(n)
y=fx+eps
px=exp(fx)
px=px/(1+px)
ly=rbinom(n=length(px),prob=px,size=1)
system.time(fit1<-glmnet(sx,y))
system.time(fit2n<-glmnet(x,y))
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.