autotune_lasso: Data-adaptive automatic and fast tuning of LM with Lasso...

In autotune: Faster and more Efficient Lasso (than 'glmnet' and 'scalreg') with Data-Driven Tuning

Data-adaptive automatic and fast tuning of LM with Lasso regularization

Description

Fits a linear model via alternative optimization of penalized gaussian maximum likelihood which is a biconvex function of regression coefficients \beta and noise variance \sigma^2. The regularization path of autotune_lasso quickly picks out a good lambda for Lasso and then returns the corresponding linear fit along with various attributes related to the fit.

Usage

autotune_lasso(
  x,
  y,
  alpha = 0.01,
  standardize = TRUE,
  standardize_response = TRUE,
  intercept = TRUE,
  active = FALSE,
  trace_it = FALSE,
  tolerance = 1e-04,
  beta_tolerance = 0.001,
  iter_max = 30,
  beta_iter_max = 40,
  active_iter_max = 5,
  PR_norm_l2 = FALSE,
  ...
)

Arguments

x	Matrix of predictors, with one column for each predictor. Dimension will be nobs \times nvars; so each row is a new observation and nvars > 1.
y	Vector of responses.
alpha	Default 0.01, significance level of sequential F-tests used for estimation of support set.
standardize	Logical flag for standardization of all variables in x, prior to fitting the model sequence. The coefficients are always returned on the original scale. Default value is TRUE.
standardize_response	Logical flag for demeaning the reponse variable y. Default value is TRUE.
intercept	Should intercept(s) be fitted (default=TRUE) or set to zero (FALSE).
active	Should active set selection be used (TRUE) or avoided (default = FALSE). It is under experimentation phase, use it with care.
trace_it	Logical input, default FALSE; if TRUE prints out the iteration number while running, useful for big datasets that take a long time to fit.
tolerance	Numeric input for an additional stopping criteria on the coordinate descent when sigma is updating. Stops that coordinate descent when the relative change between successive iterates of coefficients' estimates is less than the tolerance. Default value is 10^{-4}.
beta_tolerance	Numeric input for the stopping criteria on the coordinate descent when sigma is not updating. Stops that coordinate descent when the relative change between successive iterates of coefficients' estimates is less than the beta_tolerance. Default value is 10^{-3}.
iter_max	Maximum number of iterations of coordinate descent allowed when sigma is updating. Default value is 30.
beta_iter_max	Maximum number of iterations of coordinate descent allowed when sigma is not updating. Default value is 40.
active_iter_max	If active = TRUE, maximum number of times the active set is updated as per the violations of KKT conditions. Default value is 5.
PR_norm_l2	Logical flag to whether use the l2 norm of partial residuals for ordering them instead of the default l1 norm.
...	Additional arguments passed to the internal fitting routine.

Value

A list with various intermediate and final outputs produced by autotune Lasso in its regularization path.

beta	Final estimates of regression coefficients.
a0	Intercept of the fit.
lambda	Final thresholding value \lambda = \lambda_0\hat\sigma^2 used in the coordinate descent after noise variance estimate \hat\sigma^2 has converged.
sigma_sq	Final estimate of noise variance \sigma^2
nobs	Number of observations.
nvars	Number of variables.
CD.path.details	A list of additional details about the coordinate descent path taken by autotune Lasso: sorted_predictors Decreasing ordering of predictors in terms of their contribution to predicting the response values. sigma_sq_seq A no_of_iterations-length sequence of noise variance estimates \sigma^2. beta_matrix A (length(sigma_sq_seq) + 1) \times nvars matrix of estimated coefficients. no_of_iter_before_lambda_conv Number of coordinate descent iterations performed before the noise variance estimate \hat{\sigma}^2 converged. no_of_iter_after_lambda_conv After the noise variance estimate \hat{\sigma}^2 has converged, the number of coordinate descent iterations required for coefficients \hat\beta to converge. no_of_iterations Total number of coordinate descent iterations implemented by autotune Lasso. lambda0 Value of \lambda_0 used in autotune Lasso. Refer to the original paper for details. support_set Final set of predictors included in the support set for noise variance estimation by autotune Lasso. count_sig_beta A no_of_iterations-length vector containing the support set sizes across the coordinate descent iterations while the noise variance estimate is being updated. null_support Boolean output indicating whether the final support set estimate is a null set. If so, autotune_lasso uses the support set estimate from the previous iteration to obtain the final noise variance estimate \sigma^2. active_iterations If active = TRUE, the number of times the active set is updated according to violations of the KKT conditions. active_set_sizes An active_iterations-length vector containing the active-set sizes used for coordinate descent across the active-set iterations.

Examples

library(autotune)
set.seed(10)
n <- 80
p <- 400
s <- 5
snr <- 4
betatrue <- c(rep(1,s), rep(0, p - s))
x <- matrix(rnorm(n * p), ncol = p)
error.sd <- sqrt((betatrue %*% betatrue)/snr)
err <- rnorm(n, sd = error.sd)
y <- x %*% betatrue + err
ans <- autotune_lasso(x, y, trace_it = TRUE)
b <- betatrue
# The Predictors which are actually significant:
which(b != 0)
# The Predictors which had nonzero estmated coefficients:
which(ans$beta != 0)
# Top 10 predictors X_i's in the ranking of X_i's given by autotune:
ans$CD.path.details$sorted_predictors[1:10]
# Cardinality of autotune's estimated support set in each CD iteration before lambda converged:
ans$CD.path.details$count_sig_beta
# Sigma estimates in each CD iteration:
ans$CD.path.details$sigma_sq_seq
# Empirical noise variance:
var(err)

autotune documentation built on Aug. 21, 2026, 5:18 p.m.