ENET: Elastic Net Regularization

Description Usage Arguments Value Iteration History Author(s) References See Also Examples

Description

Elastic Net regularization is a combination of \ell_2 stability and \ell_1 sparsity constraint simulatenously solving the following,

\textrm{min}_x ~ \frac{1}{2}\|Ax-b\|_2^2 + λ_1 \|x\|_1 + λ_2 \|x\|_2^2

with nonnegative constraints λ_1 and λ_2. Note that if both lambda values are 0, it reduces to least-squares solution.

Usage

1
2
admm.enet(A, b, lambda1 = 1, lambda2 = 1, rho = 1, abstol = 1e-04,
  reltol = 0.01, maxiter = 1000)

Arguments

A

an (m\times n) regressor matrix

b

a length-m response vector

lambda1

a regularization parameter for \ell_1 term

lambda2

a regularization parameter for \ell_2 term

rho

an augmented Lagrangian parameter

abstol

absolute tolerance stopping criterion

reltol

relative tolerance stopping criterion

maxiter

maximum number of iterations

Value

a named list containing

x

a length-n solution vector

history

dataframe recording iteration numerics. See the section for more details.

Iteration History

When you run the algorithm, output returns not only the solution, but also the iteration history recording following fields over iterates,

objval

object (cost) function value

r_norm

norm of primal residual

s_norm

norm of dual residual

eps_pri

feasibility tolerance for primal feasibility condition

eps_dual

feasibility tolerance for dual feasibility condition

In accordance with the paper, iteration stops when both r_norm and s_norm values become smaller than eps_pri and eps_dual, respectively.

Author(s)

Xiaozhi Zhu

References

\insertRef

zou_regularization_2005ADMM

See Also

admm.lasso

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
## generate underdetermined design matrix
m = 50
n = 100
p = 0.1   # percentange of non-zero elements

x0 = matrix(Matrix::rsparsematrix(n,1,p))
A  = matrix(rnorm(m*n),nrow=m)
for (i in 1:ncol(A)){
  A[,i] = A[,i]/sqrt(sum(A[,i]*A[,i]))
}
b = A%*%x0 + sqrt(0.001)*matrix(rnorm(m))

## run example with both regularization values = 1
output = admm.enet(A, b, lambda1=1, lambda2=1)

## report convergence plot
niter  = length(output$history$s_norm)
par(mfrow=c(1,3))
plot(1:niter, output$history$objval, "b", main="cost function")
plot(1:niter, output$history$r_norm, "b", main="primal residual")
plot(1:niter, output$history$s_norm, "b", main="dual residual")

ADMM documentation built on May 29, 2018, 5:04 p.m.

Related to ENET in ADMM...