isgd: Implicit Stochastic Gradient Descent Optimizer

In bamlss: Bayesian Additive Models for Location, Scale, and Shape (and Beyond)

Implicit Stochastic Gradient Descent Optimizer

Description

This optimizer performs an implicit stochastic gradient descent algorithm. It is mainly used within a bamlss call.

Usage

opt_isgd(x, y, family, weights = NULL, offset = NULL,
     gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
     CFun = function(beta) diag(length(beta)),
     start = NULL, i.state = 0)

Arguments

x	For function boost() the x list, as returned from function bamlss.frame, holding all model matrices and other information that is used for fitting the model.
y	The model response, as returned from function bamlss.frame.
family	A bamlss family object, see family.bamlss.
weights	Prior weights on the data, as returned from function bamlss.frame.
offset	Can be used to supply model offsets for use in fitting, returned from function bamlss.frame.
gammaFun	Function specifying the step length.
shuffle	Should the data be shuffled?
CFun	Hessian approximating function.
start	Vector of starting values.
i.state	Added to gammaFUN().

Details

tpf

Value

For function opt_isgd() a list containing the following objects:

fitted.values	A named list of the fitted values based on the last iteration of the modeled parameters of the selected distribution.
parameters	A matrix, each row corresponds to the parameter values of one iteration.
sgd.summary	The summary of the stochastic gradient descent algorithm which can be printed and plotted.

Warning

CAUTION: Arguments weights and offset are not implemented yet!

Note

Motivated by the lecture 'Regression modelling with large data sets' given by Ioannis Kosmidis in Innsbruck, January 2017.

Author(s)

Thorsten Simon

References

Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on stochastic approximations: Classical results and new insights. Statistics and Computing, 25, no. 4, 781–795. doi: 10.1007/s11222-015-9560-y

See Also

bamlss.frame, bamlss

Examples

## Not run: 
set.seed(111)
d <- GAMart(n = 10000)
f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE)
plot(b, ask = F)

## loop over observations a 2nd time
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE, start = parameters(b),
            i.state = b$model.stats$optimizer$sgd.summary$i.state)
plot(b, ask = F)

## try differeent gammaFuns, e.g.,
# gammaFun <- function(i) .3/sqrt((1+i)) + 0.001

## testing some families
f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b2 <- bamlss(f2, data = d, optimizer = opt_isgd, sampler = FALSE, family = "binomial")

f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b3 <- bamlss(f3, data = d, optimizer = opt_isgd, sampler = FALSE, family = "cnorm")

## End(Not run)

bamlss documentation built on Oct. 11, 2024, 5:07 p.m.