lasso: Lasso Smooth Constructor

In bamlss: Bayesian Additive Models for Location, Scale, and Shape (and Beyond)

Lasso Smooth Constructor

Description

Smooth constructors and optimizer for Lasso penalization with bamlss. The penalization is based on a Taylor series approximation of the Lasso penalty.

Usage

## Smooth constructor function.
la(formula, type = c("single", "multiple"), ...)

## Single Lasso smoothing parameter optimizer.
opt_lasso(x, y, start = NULL, adaptive = TRUE, lower = 0.001, upper = 1000,
  nlambda = 100, lambda = NULL,  multiple = FALSE, verbose = TRUE,
  digits = 4, flush = TRUE, nu = NULL, stop.nu = NULL,
  ridge = .Machine$double.eps^0.5, zeromodel = NULL, ...)

lasso(x, y, start = NULL, adaptive = TRUE, lower = 0.001, upper = 1000,
  nlambda = 100, lambda = NULL,  multiple = FALSE, verbose = TRUE,
  digits = 4, flush = TRUE, nu = NULL, stop.nu = NULL,
  ridge = .Machine$double.eps^0.5, zeromodel = NULL, ...)

## Lasso transformation function to set
## adaptive weights from an unpenalized model.
lasso_transform(x, zeromodel, nobs = NULL, ...)

## Plotting function for opt_lasso() optimizer.
lasso_plot(x, which = c("criterion", "parameters"),
  spar = TRUE, model = NULL, name = NULL, mstop = NULL,
  retrans = FALSE, color = NULL, show.lambda = TRUE,
  labels = NULL, digits = 2, ...)

## Extract optimum stopping iteration for opt_lasso() optimizer.
## Based on the minimum of the information criterion.
lasso_stop(x)

## Extract retransformed Lasso coefficients.
lasso_coef(x, ...)

Arguments

formula	A formula like ~ x1 + x2 + ... + xk of variables which should be penalized with Lasso.
type	Should one single penalty parameter be used or multiple parameters, one for each covariate in formula.
x	For function opt_lasso() and lasso_transform() the x list, as returned from function bamlss.frame, holding all model matrices and other information that is used for fitting the model. For the plotting function and lasso_stop()/lasso_coef() the corresponding bamlss object fitted with the opt_lasso() optimizer.
y	The model response, as returned from function bamlss.frame.
start	A vector of starting values. Note, Lasso smoothing parameters will be dropped.
adaptive	Should adaptive weights be used for fused Lasso terms?
lower	Numeric. The minimum lambda value.
upper	Numeric. The maximum lambda value.
nlambda	Integer. The number of smoothing parameters for which coefficients should be estimated, i.e., the vector of smoothing parameters is build up as a sequence from lower to upper with length nlambda.
lambda	Numeric. A sequence/vector of lambda parameters that should be used.
multiple	Logical. Should the lambda grid be exapnded to search for multiple lambdas, one for each distributional parameter.
verbose	Print information during runtime of the algorithm.
digits	Set the digits for printing when verbose = TRUE. If the optimum lambda value is plotted, the number of decimal decimal places to be used within lasso_plot().
flush	use flush.console for displaying the current output in the console.
nu	Numeric or logical. Defines the step length for parameter updating of a model term, useful when the algorithm encounters convergence problems. If nu = TRUE the step length parameter is optimized for each model term in each iteration of the backfitting algorithm.
stop.nu	Integer. Should step length reduction be stopped after stop.nu iterations of the Lasso algorithm?
ridge	A ridge penalty parameter that should be used when finding adaptive weights, i.e., parameters from an unpenalized model. The ridge penalty is used to stabilize the estimation in complex models.
zeromodel	A model containing the unpenalized parameters, e.g., for each la() terms one can place a simple ridge penalty with la(x, ridge = TRUE, sp = 0.1). This way it is possible to find the unpenalized parameters that can be used as adaptive weights for fusion penalties.
nobs	Integer, number of observations of the data used for modeling. If not supplied nobs is taken from the number of rows from the model term design matrices.
which	Which of the two provided plots should be created, character or integer 1 and 2.
spar	Should graphical parameters be set by the plotting function?
model	Character selecting for which model the plot shpuld be created.
name	Character, the name of the coefficient group that should be plotted. Note that the string provided in name will be removed from the labels on the 4th axis.
mstop	Integer vector, defines the path length to be plotted.
retrans	Logical, should coefficients be re-transformed before plotting?
color	Colors or color function that creates colors for the group paths.
show.lambda	Logical. Should the optimum value of the penalty parameter lambda be shown?
labels	A character string of labels that should be used on the 4 axis.
...	Arguments passed to the subsequent smooth constructor function. lambda controls the starting value of the penalty parameter, const the constant that is added within the penalty approximation. Moreover, fuse = 1 enforces nominal fusion of categorical variables and fuse = 2 ordered fusion within la() Note that la() terms with and without fusion should not be mixed when using the opt_lasso() optimizer function. For the optimizer opt_lasso() arguments passed to function bfit.

Value

For function la(), similar to function s a simple smooth specification object.

For function opt_lasso() a list containing the following objects:

fitted.values	A named list of the fitted values based on the last lasso iteration of the modeled parameters of the selected distribution.
parameters	A matrix, each row corresponds to the parameter values of one boosting iteration.
lasso.stats	A matrix containing information about the log-likelihood, log-posterior and the information criterion for each lambda.

References

Andreas Groll, Julien Hambuckers, Thomas Kneib, and Nikolaus Umlauf (2019). Lasso-type penalization in the framework of generalized additive models for location, scale and shape. Computational Statistics & Data Analysis. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.csda.2019.06.005")}

Oelker Margreth-Ruth and Tutz Gerhard (2015). A uniform framework for combination of penalties in generalized structured models. Adv Data Anal Classif. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11634-015-0205-y")}

See Also

s, smooth.construct

Examples

## Not run: ## Simulated fusion Lasso example.
bmu <- c(0,0,0,2,2,2,4,4,4)
bsigma <- c(0,0,0,-2,-2,-2,-1,-1,-1)
id <- factor(sort(rep(1:length(bmu), length.out = 300)))

## Response.
set.seed(123)
y <- bmu[id] + rnorm(length(id), sd = exp(bsigma[id]))

## Estimate model:
## fuse=1 -> nominal fusion,
## fuse=2 -> ordinal fusion,
## first, unpenalized model to be used for adaptive fusion weights.
f <- list(y ~ la(id,fuse=2,fx=TRUE), sigma ~ la(id,fuse=1,fx=TRUE))
b0 <- bamlss(f, sampler = FALSE)

## Model with single lambda parameter.
f <- list(y ~ la(id,fuse=2), sigma ~ la(id,fuse=1))
b1 <- bamlss(f, sampler = FALSE, optimizer = opt_lasso,
  criterion = "BIC", zeromodel = b0)

## Plot information criterion and coefficient paths.
lasso_plot(b1, which = 1)
lasso_plot(b1, which = 2)
lasso_plot(b1, which = 2, model = "mu", name = "mu.s.la(id).id")
lasso_plot(b1, which = 2, model = "sigma", name = "sigma.s.la(id).id")

## Extract coefficients for optimum Lasso parameter.
coef(b1, mstop = lasso_stop(b1))

## Predict with optimum Lasso parameter.
p1 <- predict(b1, mstop = lasso_stop(b1))

## Full MCMC, needs lasso_transform() to assign the
## adaptive weights from unpenalized model b0.
b2 <- bamlss(f, optimizer = FALSE, transform = lasso_transform,
  zeromodel = b0, nobs = length(y), start = coef(b1, mstop = lasso_stop(b1)),
  n.iter = 4000, burnin = 1000)
summary(b2)
plot(b2)

ci <- confint(b2, model = "mu", pterms = FALSE, sterms = TRUE)
lasso_plot(b1, which = 2, model = "mu", name = "mu.s.la(id).id", spar = FALSE)
for(i in 1:8) {
  abline(h = ci[i, 1], lty = 2, col = "red")
  abline(h = ci[i, 2], lty = 2, col = "red")
}

## End(Not run)

bamlss documentation built on Sept. 26, 2025, 3 p.m.