boost: Boosting BAMLSS

In bamlss: Bayesian Additive Models for Location, Scale, and Shape (and Beyond)

Boosting BAMLSS

Description

Optimizer functions for gradient and likelihood boosting with bamlss. In each boosting iteration the function selects the model term with the largest contribution to the log-likelihood, AIC or BIC.

Usage

## Gradient boosting optimizer.
opt_boost(x, y, family, weights = NULL,
  offset = NULL, nu = 0.1, nu.adapt = TRUE, df = 4, maxit = 400,
  mstop = NULL, maxq = NULL, qsel.splitfactor = FALSE,
  verbose = TRUE, digits = 4, flush = TRUE,
  eps = .Machine$double.eps^0.25,
  nback = NULL, plot = TRUE, initialize = TRUE,
  stop.criterion = NULL, select.type = 1,
  force.stop = TRUE, hatmatrix = !is.null(stop.criterion),
  reverse.edf = FALSE, approx.edf = FALSE,
  always = FALSE, ...)

boost(x, y, family, weights = NULL,
  offset = NULL, nu = 0.1, nu.adapt = TRUE, df = 4, maxit = 400,
  mstop = NULL, maxq = NULL, qsel.splitfactor = FALSE,
  verbose = TRUE, digits = 4, flush = TRUE,
  eps = .Machine$double.eps^0.25,
  nback = NULL, plot = TRUE, initialize = TRUE,
  stop.criterion = NULL, select.type = 1,
  force.stop = TRUE, hatmatrix = !is.null(stop.criterion),
  reverse.edf = FALSE, approx.edf = FALSE,
  always = FALSE, ...)

## Modified likelihood based boosting.
opt_boostm(x, y, family, offset = NULL,
  nu = 0.1, df = 3, maxit = 400, mstop = NULL,
  verbose = TRUE, digits = 4, flush = TRUE,
  eps = .Machine$double.eps^0.25, plot = TRUE,
  initialize = TRUE, stop.criterion = "BIC",
  force.stop = !is.null(stop.criterion),
  do.optim = TRUE, always = FALSE, ...)

boostm(x, y, family, offset = NULL,
  nu = 0.1, df = 3, maxit = 400, mstop = NULL,
  verbose = TRUE, digits = 4, flush = TRUE,
  eps = .Machine$double.eps^0.25, plot = TRUE,
  initialize = TRUE, stop.criterion = "BIC",
  force.stop = !is.null(stop.criterion),
  do.optim = TRUE, always = FALSE, ...)

## Boosting summary extractor.
boost_summary(object, ...)

## Plot all boosting paths.
boost_plot(x, which = c("loglik", "loglik.contrib", "parameters",
  "aic", "bic", "user"), intercept = TRUE, spar = TRUE, mstop = NULL,
  name = NULL, drop = NULL, labels = NULL, color = NULL, ...)

## Boosting summary printing and plotting.
## S3 method for class 'boost_summary'
print(x, summary = TRUE, plot = TRUE,
  which = c("loglik", "loglik.contrib"), intercept = TRUE,
  spar = TRUE, ...)
## S3 method for class 'boost_summary'
plot(x, ...)

## Model frame for out-of-sample selection.
boost_frame(formula, train, test, family = "gaussian", ...)

Arguments

x	For function opt_boost() the x list, as returned from function bamlss.frame, holding all model matrices and other information that is used for fitting the model. For the plotting function the corresponding bamlss object fitted with the opt_boost() optimizer.
y	The model response, as returned from function bamlss.frame.
family	A bamlss family object, see family.bamlss.
weights	Prior weights on the data, as returned from function bamlss.frame.
offset	Can be used to supply model offsets for use in fitting, returned from function bamlss.frame.
nu	Numeric, between [0, 1], controls the step size, i.e., the amount that should be added to model term parameters.
nu.adapt	Logical. If set to TRUE (default) step size nu is divided by 2, if current boosting iteration did not improve the loglikelihood.
df	Integer, defines the initial degrees of freedom that should be assigned to each smooth model term. May also be a named vector, the names must match the model term labels, e.g., as provided in summary.bamlss.
maxit	Integer, the maximum number of boosting iterations.
mstop	For convenience, overwrites maxit.
maxq	Integer, defines the maximum number of selected base-learners. The algorithm stops if this numer is exceeded.
qsel.splitfactor	Logical, if set to TRUE dummy variables of categorical predictors are counted individually.
name	Character, the name of the coefficient (group) that should be plotted. Note that the string provided in name will be removed from the labels on the 4th axis.
drop	Character, the name of the coefficient (group) that should not be plotted.
labels	A character string of labels that should be used on the 4 axis.
color	Colors or color function that creates colors for the (group) paths.
verbose	Print information during runtime of the algorithm.
digits	Set the digits for printing when verbose = TRUE.
flush	use flush.console for displaying the current output in the console.
eps	The tolerance used as stopping mechanism, see argument nback.
nback	Integer. If nback is not NULL, then the algorithm stops if the the change in the log-likelihood of the last nback iterations is smaller or equal to eps. If maxit = NULL the maximum number of iterations is set to 10000.
plot	Should the boosting summary be printed and plotted?
initialize	Logical, should intercepts be initialized?
stop.criterion	Character, selects the information criterion that should be used to determine the optimum number of boosting iterations. Either "AIC" or "BIC" is possible. Note that this feature requires to compute hat-matrices for each distributional parameter, therefore, the routine may be slow and computer storage intensive.
select.type	Should model terms be selected by the log-likelihood contribution, select.type = 1, or by the corresponding stop.criterion, select.type = 2.
force.stop	Logical, should the algorithm stop if the information criterion increases?
do.optim	Logical. Should smoothing parameters be optimized in each boosting iteration?
hatmatrix	Logical, if set to TRUE the hat-matrices for each distributional parameter will be computed. The hat-matrices are used to determine the effective (equivalent) degrees of freedom in each boosting iteration, i.e., it is possible to compute information criteria like the AIC or BIC for selecting the optimum number of boosting iterations.
reverse.edf	Logical. Instead of computing degrees of freedom with hat-matrices, the actual smoothing parameters are reverse engineered to compute the corresponding actual smoother matrix. Note that this option is still experimental.
approx.edf	Logical. Another experimental and fast approximation of the degrees of freedom.
always	Logical or character. Should the intercepts forced to be updated in each boosting iteration? If always = TRUE each intercept of each distributional parameter is updated, if always = "best" only the intercept corresponding to the distributional of the best fitting model term is updated.
object	A bamlss object that was fitted using opt_boost().
summary	Should the summary be printed?
which	Which of the three provided plots should be created?
intercept	Should the coefficient paths of intercepts be dropped in the plot?
spar	Should graphical parmeters be set with par?
formula	See bamlss.frame.
train, test	Data frames used for training and testing the model..
...	For function opt_boost(), arguments passed to bamlss.engine.setup. for function boost_summary() arguments passed to function print.boost_summary().

Value

For function boost_summary() a list containing information on selection frequencies etc. For function opt_boost() and opt_boostm() a list containing the following objects:

fitted.values	A named list of the fitted values based on the last boosting iteration of the modeled parameters of the selected distribution.
parameters	A matrix, each row corresponds to the parameter values of one boosting iteration.
boost_summary	The boosting summary which can be printed and plotted.

WARNINGS

The function does not take care of variable scaling for the linear parts! This must be done by the user, e.g., one option is to use argument scale.d in function bamlss.frame, which uses scale.

Function opt_boost() does not select the optimum stopping iteration! The modified likelihood based algorithm implemented in function opt_boostm() is still experimental!

See Also

bamlss.frame, bamlss

Examples

## Not run: ## Simulate data.
set.seed(123)
d <- GAMart()

## Estimate model.
f <- num ~ x1 + x2 + x3 + lon + lat +
  s(x1) + s(x2) + s(x3) + s(lon) + s(lat) + te(lon,lat)

b <- bamlss(f, data = d, optimizer = opt_boost,
  sampler = FALSE, scale.d = TRUE, nu = 0.01,
  maxit = 1000, plot = FALSE)

## Plot estimated effects.
## plot(b)

## Print and plot the boosting summary.
boost_summary(b, plot = FALSE)
## boost_plot(b, which = 1)
## boost_plot(b, which = 2)
## boost_plot(b, which = 3, name = "mu.s.te(lon,lat).")

## Extract estimated parameters for certain
## boosting iterations.
parameters(b, mstop = 1)
parameters(b, mstop = 100)

## Also works with predict().
head(do.call("cbind", predict(b, mstop = 1)))
head(do.call("cbind", predict(b, mstop = 100)))

## Another example using the modified likelihood
## boosting algorithm.
f <- list(
  num ~ x1 + x2 + x3 + lon + lat +
    s(x1) + s(x2) + s(x3) + s(lon) + s(lat) + te(lon,lat),
  sigma ~ x1 + x2 + x3 + lon + lat +
    s(x1) + s(x2) + s(x3) + s(lon) + s(lat) + te(lon,lat)
)

b <- bamlss(f, data = d, optimizer = opt_boostm,
  sampler = FALSE, scale.d = TRUE, nu = 0.05,
  maxit = 400, stop.criterion = "AIC", force.stop = FALSE)

## Plot estimated effects.
## plot(b)

## Plot AIC and log-lik contributions.
## boost_plot(b, "AIC")
## boost_plot(b, "loglik.contrib")

## Out-of-sample selection of model terms.
set.seed(123)
d <- GAMart(n = 5000)

## Split data into training and testing
i <- sample(1:2, size = nrow(d), replace = TRUE)
dtest <- subset(d, i == 1)
dtrain <- subset(d, i == 2)

## Model formula
f <- list(
  num ~ s(x1) + s(x2) + s(x3),
  sigma ~ s(x1) + s(x2) + s(x3)
)

## Create model frame for out-of-sample selection.
sm <- boost_frame(f, train = dtrain, test = dtest, family = "gaussian")

## Out-of-sample selection function.
sfun <- function(parameters) {
  sm$parameters <- parameters
  p <- predict(sm, type = "parameter")
  -1 * sum(sm$family$d(dtest$num, p, log = TRUE))
}

## Start boosting with out-of-sample negative
## log-likelihood selection of model terms.
b <- bamlss(f, data = dtrain, sampler = FALSE, optimizer = opt_boost,
  selectfun = sfun, always = "best")

## Plot curve of negative out-of-sample log-likelihood.
## boost_plot(b, which = "user")

## End(Not run)

bamlss documentation built on Oct. 11, 2024, 5:07 p.m.