# ENET: Elastic Net Regularization In ADMM: Algorithms using Alternating Direction Method of Multipliers

## Description

Elastic Net regularization is a combination of \ell_2 stability and \ell_1 sparsity constraint simulatenously solving the following,

\textrm{min}_x ~ \frac{1}{2}\|Ax-b\|_2^2 + λ_1 \|x\|_1 + λ_2 \|x\|_2^2

with nonnegative constraints λ_1 and λ_2. Note that if both lambda values are 0, it reduces to least-squares solution.

## Usage

 1 2 admm.enet(A, b, lambda1 = 1, lambda2 = 1, rho = 1, abstol = 1e-04, reltol = 0.01, maxiter = 1000) 

## Arguments

 A an (m\times n) regressor matrix b a length-m response vector lambda1 a regularization parameter for \ell_1 term lambda2 a regularization parameter for \ell_2 term rho an augmented Lagrangian parameter abstol absolute tolerance stopping criterion reltol relative tolerance stopping criterion maxiter maximum number of iterations

## Value

a named list containing

x

a length-n solution vector

history

dataframe recording iteration numerics. See the section for more details.

## Iteration History

When you run the algorithm, output returns not only the solution, but also the iteration history recording following fields over iterates,

objval

object (cost) function value

r_norm

norm of primal residual

s_norm

norm of dual residual

eps_pri

feasibility tolerance for primal feasibility condition

eps_dual

feasibility tolerance for dual feasibility condition

In accordance with the paper, iteration stops when both r_norm and s_norm values become smaller than eps_pri and eps_dual, respectively.

Xiaozhi Zhu

## References

\insertRef

admm.lasso
  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ## generate underdetermined design matrix m = 50 n = 100 p = 0.1 # percentange of non-zero elements x0 = matrix(Matrix::rsparsematrix(n,1,p)) A = matrix(rnorm(m*n),nrow=m) for (i in 1:ncol(A)){ A[,i] = A[,i]/sqrt(sum(A[,i]*A[,i])) } b = A%*%x0 + sqrt(0.001)*matrix(rnorm(m)) ## run example with both regularization values = 1 output = admm.enet(A, b, lambda1=1, lambda2=1) ## report convergence plot niter = length(output$history$s_norm) par(mfrow=c(1,3)) plot(1:niter, output$history$objval, "b", main="cost function") plot(1:niter, output$history$r_norm, "b", main="primal residual") plot(1:niter, output$history$s_norm, "b", main="dual residual")