l0ara: fit a generalized linear model with l0 penalty

In l0ara: Sparse Generalized Linear Model with L0 Approximation for Feature Selection

Description

An adaptive ridge algorithm for feature selection with L0 penalty.

Usage

l0ara(x, y, family, lam, standardize, maxit, eps)

Arguments

x	Input matrix, of dimension nobs x nvars; each row is an observation vector.
y	Response variable. Quantitative for family="gaussian"; positive quantitative for family="gamma" or family="inv.gaussian" ; a factor with two levels for family="logit"; non-negative counts for family="poisson".
family	Response type(see above).
lam	A user supplied lambda value. If you have a lam sequence, use cv.l0ara first to select optimal tunning and then refit with lam.min . To use AIC, set lam=2; to use BIC, set lam=log(n).
standardize	Logical flag for data normalization. If standardize=TRUE(default), independent variables in the design matrix x will be standardized with mean 0 and standard deviation 1.
maxit	Maximum number of passes over the data for lambda. Default value is 1e3.
eps	Convergence threshold. Default value is 1e-4.

Details

The sequence of models indexed by the parameter lambda is fit using adptive ridge algorithm. The objective function for generalized linear models (including family above) is defined to be

-(log likelihood)+(λ/2)*|β|_0

|β|_0 is the number of non-zero elements in β. To select the "best" model with AIC or BIC criterion, let lambda to be 2 or log(n). This adaptive ridge algorithm is developed to approximate L0 penalized generalized linear models with sequential optimization and is efficient for high-dimensional data.

Value

An object with S3 class "l0ara" containing:

beta	A vector of coefficients
df	Number of nonzero coefficients
iter	Number of iterations
lambda	The lambda used
x	Design matrix
y	Response variable

Author(s)

Wenchuan Guo <[e-mail]>, Shujie Ma <[e-mail]>, Zhenqiu Liu <[e-mail]>

See Also

cv.l0ara, predict.l0ara, coef.l0ara, plot.l0ara methods.

Examples

# Linear regression
# Generate design matrix and response variable
n <- 100
p <- 40
x <- matrix(rnorm(n*p), n, p)
beta <- c(1,0,2,3,rep(0,p-4))
noise <- rnorm(n)
y <- x%*%beta+noise
# fit sparse linear regression using BIC 
res.gaussian <- l0ara(x, y, family="gaussian", log(n))

# predict for new observations
print(res.gaussian)
predict(res.gaussian, newx=matrix(rnorm(3,p),3,p))
coef(res.gaussian)

# Logistic regression
# Generate design matrix and response variable
n <- 100
p <- 40
x <- matrix(rnorm(n*p), n, p)
beta <- c(1,0,2,3,rep(0,p-4))
prob <- exp(x%*%beta)/(1+exp(x%*%beta))
y <- rbinom(n, rep(1,n), prob)
# fit sparse logistic regression
res.logit <- l0ara(x, y, family="logit", 0.7)

# predict for new observations
print(res.logit)
predict(res.logit, newx=matrix(rnorm(3,p),3,p))
coef(res.logit)

# Poisson regression
# Generate design matrix and response variable
n <- 100
p <- 40
x <- matrix(rnorm(n*p), n, p)
beta <- c(1,0,0.5,0.3,rep(0,p-4))
mu <- exp(x%*%beta)
y <- rpois(n, mu)
# fit sparse Poisson regression using AIC
res.pois <- l0ara(x, y, family="poisson", 2)

# predict for new observations
print(res.pois)
predict(res.pois, newx=matrix(rnorm(3,p),3,p))
coef(res.pois)

Example output

Lambda used :  4.60517 
Model :  Linear regression 
Iterations :  8 
Degree of freedom :  3 
[1] 248.4098 244.2392 228.3004
Intercept        X1        X2        X3        X4        X5        X6        X7 
0.8112056 0.0000000 2.0691321 3.1081400 0.0000000 0.0000000 0.0000000 0.0000000 
       X8        X9       X10       X11       X12       X13       X14       X15 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X16       X17       X18       X19       X20       X21       X22       X23 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X24       X25       X26       X27       X28       X29       X30       X31 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X32       X33       X34       X35       X36       X37       X38       X39 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
Lambda used :  0.7 
Model :  Logistic regression 
Iterations :  27 
Degree of freedom :  4 
[1] 236.2007 225.3518 225.5914
 Intercept         X1         X2         X3         X4         X5         X6 
 1.1178504  0.0000000  2.1147630  3.1209154  0.0000000  0.0000000  0.0000000 
        X7         X8         X9        X10        X11        X12        X13 
 0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000 
       X14        X15        X16        X17        X18        X19        X20 
 0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000 
       X21        X22        X23        X24        X25        X26        X27 
 0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000 
       X28        X29        X30        X31        X32        X33        X34 
 0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000  0.0000000 
       X35        X36        X37        X38        X39 
 0.0000000  0.0000000  0.0000000  0.0000000 -0.6188048 
Lambda used :  2 
Model :  Poisson regression 
Iterations :  14 
Degree of freedom :  3 
[1] 75.37462 76.55928 73.73387
Intercept        X1        X2        X3        X4        X5        X6        X7 
1.1402179 0.0000000 0.4021105 0.3023030 0.0000000 0.0000000 0.0000000 0.0000000 
       X8        X9       X10       X11       X12       X13       X14       X15 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X16       X17       X18       X19       X20       X21       X22       X23 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X24       X25       X26       X27       X28       X29       X30       X31 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
      X32       X33       X34       X35       X36       X37       X38       X39 
0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 

l0ara documentation built on Feb. 6, 2020, 5:17 p.m.