ADMMsigma: Penalized precision matrix estimation via ADMM
In ADMMsigma: Penalized Precision Matrix Estimation via ADMM

Description Usage Arguments Details Value Author(s) References See Also Examples

Penalized precision matrix estimation using the ADMM algorithm. Consider the case where X_{1}, ..., X_{n} are iid N_{p}(μ, Σ) and we are tasked with estimating the precision matrix, denoted Ω \equiv Σ^{-1}. This function solves the following optimization problem:

Objective:: \hat{Ω}_{λ} = \arg\min_{Ω \in S_{+}^{p}} ≤ft\{ Tr≤ft(SΩ\right) - \log \det≤ft(Ω \right) + λ≤ft[\frac{1 - α}{2}≤ft\| Ω \right|_{F}^{2} + α≤ft\| Ω \right\|_{1} \right] \right\}

where 0 ≤q α ≤q 1, λ > 0, ≤ft\|\cdot \right\|_{F}^{2} is the Frobenius norm and we define ≤ft\|A \right\|_{1} = ∑_{i, j} ≤ft| A_{ij} \right|. This elastic net penalty is identical to the penalty used in the popular penalized regression package glmnet. Clearly, when α = 0 the elastic-net reduces to a ridge-type penalty and when α = 1 it reduces to a lasso-type penalty.

ADMMsigma(X = NULL, S = NULL, nlam = 10, lam.min.ratio = 0.01,
  lam = NULL, alpha = seq(0, 1, 0.2), diagonal = FALSE, path = FALSE,
  rho = 2, mu = 10, tau.inc = 2, tau.dec = 2, crit = c("ADMM",
  "loglik"), tol.abs = 1e-04, tol.rel = 1e-04, maxit = 10000,
  adjmaxit = NULL, K = 5, crit.cv = c("loglik", "penloglik", "AIC",
  "BIC"), start = c("warm", "cold"), cores = 1, trace = c("progress",
  "print", "none"))

`X`	option to provide a nxp data matrix. Each row corresponds to a single observation and each column contains n observations of a single feature/variable.
`S`	option to provide a pxp sample covariance matrix (denominator n). If argument is `NULL` and `X` is provided instead then `S` will be computed automatically.
`nlam`	number of `lam` tuning parameters for penalty term generated from `lam.min.ratio` and `lam.max` (automatically generated). Defaults to 10.
`lam.min.ratio`	smallest `lam` value provided as a fraction of `lam.max`. The function will automatically generate `nlam` tuning parameters from `lam.min.ratio*lam.max` to `lam.max` in log10 scale. `lam.max` is calculated to be the smallest `lam` such that all off-diagonal entries in `Omega` are equal to zero (`alpha` = 1). Defaults to 1e-2.
`lam`	option to provide positive tuning parameters for penalty term. This will cause `nlam` and `lam.min.ratio` to be disregarded. If a vector of parameters is provided, they should be in increasing order. Defaults to NULL.
`alpha`	elastic net mixing parameter contained in [0, 1]. `0 = ridge, 1 = lasso`. If a vector of parameters is provided, they should be in increasing order. Defaults to grid of values `seq(0, 1, 0.2)`.
`diagonal`	option to penalize the diagonal elements of the estimated precision matrix (Ω). Defaults to `FALSE`.
`path`	option to return the regularization path. This option should be used with extreme care if the dimension is large. If set to TRUE, cores must be set to 1 and errors and optimal tuning parameters will based on the full sample. Defaults to FALSE.
`rho`	initial step size for ADMM algorithm.
`mu`	factor for primal and residual norms in the ADMM algorithm. This will be used to adjust the step size `rho` after each iteration.
`tau.inc`	factor in which to increase step size `rho`
`tau.dec`	factor in which to decrease step size `rho`
`crit`	criterion for convergence (`ADMM` or `loglik`). If `crit = loglik` then iterations will stop when the relative change in log-likelihood is less than `tol.abs`. Default is `ADMM` and follows the procedure outlined in Boyd, et al.
`tol.abs`	absolute convergence tolerance. Defaults to 1e-4.
`tol.rel`	relative convergence tolerance. Defaults to 1e-4.
`maxit`	maximum number of iterations. Defaults to 1e4.
`adjmaxit`	adjusted maximum number of iterations. During cross validation this option allows the user to adjust the maximum number of iterations after the first `lam` tuning parameter has converged (for each `alpha`). This option is intended to be paired with `warm` starts and allows for 'one-step' estimators. Defaults to NULL.
`K`	specify the number of folds for cross validation.
`crit.cv`	cross validation criterion (`loglik`, `penloglik`, `AIC`, or `BIC`). Defaults to `loglik`.
`start`	specify `warm` or `cold` start for cross validation. Default is `warm`.
`cores`	option to run CV in parallel. Defaults to `cores = 1`.
`trace`	option to display progress of CV. Choose one of `progress` to print a progress bar, `print` to print completed tuning parameters, or `none`.

For details on the implementation of 'ADMMsigma', see the website https://mgallow.github.io/ADMMsigma/articles/Details.html.

returns class object ADMMsigma which includes:

`Call`	function call.
`Iterations`	number of iterations.
`Tuning`	optimal tuning parameters (lam and alpha).
`Lambdas`	grid of lambda values for CV.
`Alphas`	grid of alpha values for CV.
`maxit`	maximum number of iterations.
`Omega`	estimated penalized precision matrix.
`Sigma`	estimated covariance matrix from the penalized precision matrix (inverse of Omega).
`Path`	array containing the solution path. Solutions will be ordered in ascending alpha values for each lambda.
`Z`	final sparse update of estimated penalized precision matrix.
`Y`	final dual update.
`rho`	final step size.
`Loglik`	penalized log-likelihood for Omega
`MIN.error`	minimum average cross validation error (cv.crit) for optimal parameters.
`AVG.error`	average cross validation error (cv.crit) across all folds.
`CV.error`	cross validation errors (cv.crit).

Matt Galloway gall0441@umn.edu

Boyd, Stephen, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, and others. 2011. 'Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers.' Foundations and Trends in Machine Learning 3 (1). Now Publishers, Inc.: 1-122. https://web.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf
Hu, Yue, Chi, Eric C, amd Allen, Genevera I. 2016. 'ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning.' Splitting Methods in Communication, Imaging, Science, and Engineering. Springer: 433-459.
Zou, Hui and Hastie, Trevor. 2005. 'Regularization and Variable Selection via the Elastic Net.' Journal of the Royal Statistial Society: Series B (Statistical Methodology) 67 (2). Wiley Online Library: 301-320.
Rothman, Adam. 2017. 'STAT 8931 notes on an algorithm to compute the Lasso-penalized Gaussian likelihood precision matrix estimator.'

plot.ADMM, RIDGEsigma

# generate data from a sparse matrix
# first compute covariance matrix
S = matrix(0.7, nrow = 5, ncol = 5)
for (i in 1:5){
 for (j in 1:5){
   S[i, j] = S[i, j]^abs(i - j)
 }
 }

# generate 100 x 5 matrix with rows drawn from iid N_p(0, S)
set.seed(123)
Z = matrix(rnorm(100*5), nrow = 100, ncol = 5)
out = eigen(S, symmetric = TRUE)
S.sqrt = out$vectors %*% diag(out$values^0.5)
S.sqrt = S.sqrt %*% t(out$vectors)
X = Z %*% S.sqrt

# elastic-net type penalty (use CV for optimal lambda and alpha)
ADMMsigma(X)

# ridge penalty (use CV for optimal lambda)
ADMMsigma(X, alpha = 0)

# lasso penalty (lam = 0.1)
ADMMsigma(X, lam = 0.1, alpha = 1)