gvcm.cat: Regularized Categorical Effects/Categorical Effect...

In gvcm.cat: Regularized Categorical Effects/Categorical Effect Modifiers/Continuous/Smooth Effects in GLMs

Description

The function fits generalized linear models with regularized categorical effects, categorical effect modifiers, continuous effects and smooth effects. The model is specified by giving a symbolic description of the linear predictor and a description of the error distribution. Estimation employs different regularization and model selection strategies. These strategies are either a penalty or a forward selection strategy employing AIC/BIC. For non-differentiable penalties, a local quadratic approximation is employed, see Oelker and Tutz (2013).

Usage

gvcm.cat(formula, data, family = gaussian, method = c("lqa", "AIC", "BIC"), 
tuning = list(lambda=TRUE, specific=FALSE, phi=0.5, grouped.fused=0.5, 
elastic=0.5, vs=0.5, spl=0.5), weights, offset, start, control, 
model = FALSE, x = FALSE, y = FALSE, plot=FALSE, ...)

pest(x, y, indices, family = gaussian, 
tuning = list(lambda=TRUE, specific=FALSE, phi=0.5, grouped.fused=0.5, 
elastic=0.5, vs=0.5, spl=0.5), weights, offset, start = NULL, 
control = cat_control(), plot=FALSE, ...)

abc(x, y, indices, family = gaussian, tuning = c("AIC", "BIC"), 
weights, offset, start, control = cat_control(), plot=FALSE, ...)

Arguments

formula	an object of class formula: a symbolic description of the model to be fitted. See details
data	a data frame, containing the variables in the model
family	a family object describing the error distribution and link function to be used in the model; this can be a character string naming a family function, a family function or the result of a call to a family function, see family for details; currently only gaussian, binomial, poisson, Gamma are working
method	fitting method; one out of "lqa", "AIC" or "BIC"; method "lqa" induces penalized estimation; it employs a PIRLS-algorithm (see Fan and Li, 2001; Oelker and Tutz, 2013). Methods "AIC" and "BIC" employ a forward selection strategy
tuning	a list; tuning parameters for penalized estimation; lambda is the scalar, overall penalty parameter; if lambda is a vector of values, these values are cross-validated; if lambda = TRUE, lambda is cross-validated on log scale between lambda.lower and lambda.upper; see cat_control. If lambda is a vector with the same length as elements in the formula and if specific equals a vector of proper length, the entries of specific are interpreted as specific tuning parameters for each entry of the formula. phi, grouped.fused, elastic, vs and spl are parameters that weigh the terms of some penalties; must be out of intervall )0,1(; the default 0.5 corresponds to equal weights
weights	an optional weight vector (for the observations)
offset	an optional offset
start	initial values for the PIRLS algorithm for method lqa
control	a list of parameters for controlling the fitting process; if emtpy, set to cat_control(); see cat_control
model	for functions gvcm.cat: a logical value indicating whether the employed model frame shall be returned or not
x, y	for function gvcm.cat: logical values indicating whether the response vector and the model matrix used in the fitting process shall be returned or not; for functions pest and abc: y must be a response vector, x a proper coded design matrix
plot	logical; if TRUE, estimates needed to plot coefficient paths are computed
indices	for pest and abc only: the to be used index argument; see function index
...	further arguments passed to or from other methods

Details

A typical formula has the form response ~ 1 + terms; where response is the response vector and terms is a series of terms which specifies a linear predictor. There are some special terms for regularized terms:

• v(x, u, n="L1", bj=TRUE) : varying coefficients enter the formula as v(x,u) where u denotes the categorical effect modifier and x the modfied covariate. A varying intercept is denoted by v(1,u). Varying coefficients with categorical effect modifiers are penalized as described in Oelker et. al. 2012. The argument bj and the element phi in argument tuning allow for the described weights.

• p(u, n="L1") : ordinal/nominal covariates u given as p(u) are penalized as described in Gertheiss and Tutz (2010). For numeric covariates, p(u) indicates a pure Lasso penalty.

• grouped(u, ...): penalizes a group of covariates with the grouped Lasso penalty of Yuan and Lin (2006); so far, working for categorical covariates only

• sp(x, knots=20, n="L2"): implents a continuous x covariate non-parametrically as f(x); f(x) is represented by centered evaluations of basis functions (cubic B-splines with number of knots = knots); for n="L2", the curvature of f(x) is penalized by a Ridge penalty; see Eilers and Marx (1996)

• SCAD(u): penalizes a covariate u with the SCAD penalty by Fan and Li (2001); for categorical covariates u, differences of coefficients are penalized by a SCAD penalty, see Gertheiss and Tutz (2010)

• elastic(u): penalizes a covariate u with the elastic net penalty by Zou and Hastie (2005); for categorical covariates u, differences of coefficients are penalized by the elastic net penalty, see Gertheiss and Tutz (2010)

If the formula contains no (varying) intercept, gvcm.cat assumes a constant intercept. There is no way to avoid an intercept.
For specials p and v, there is the special argument n: if n="L1", the absolute values in the penalty are replaced by squares of the same terms; if n="L2", the absolute values in the penalty are replaced by quadratic, Ridge-type terms; if n="L0", the absolute values in the penalty are replaced by an indicator for non-zero entries of the same terms.
For methods "AIC" and "BIC", the coefficients are not penalized but selected by a forward selection strategy whenever it makes sense; for special v(x,u), the selection strategy is described in Oelker et. al. 2012; the approach for the other specials corresponds to this idea.
For binomial families the response can also be a success/failure rate or a two-column matrix with the columns giving the numbers of successes and failures.
Function pest computes penalized estimates, that is, it implements method "lqa" (PIRLS-algorithm).
Function abc implements the forward selection strategy employing AIC/BIC.
Categorical effect modifiers and penalized categorical covariates are dummy coded as required by the penalty. If x in v(x,u) is binary, it is effect coded (first category refers to -1). Other covariates are coded like given by getOption.
There is a summary function: summary.gvcm.cat

Value

gvcm.cat returns an object of class “gvcm.cat” which inherits from class “glm” which inherits from class “lm”. An object of class “gvcm.cat” contains:

coefficients	named vector of coefficients
coefficients.reduced	reduced vector of coefficients; selected coefficients/differences of coefficients are set to zero
coefficients.refitted	refitted vector of coefficients; i.e. maximum likelihood estimate of that model containing selected covariates only; same length as coefficients.reduced
coefficients.oml	maximum likelihood estimate of the full model
residuals	deviance residuals
fitted.values	fitted mean values
rank	degrees of freedom model; for method="lqa" estimated by the trace of the generalized head matrix; for methods "AIC", "BIC" estimated like default in glm.fit
family	the family object used
linear.predictors	linear fit on link scale
deviance	scaled deviance
aic	a version of Akaike's Information Criterion; minus twice the maximized log-likelihood plus twice the rank. For binomial and Poison families the dispersion is fixed at one. For a gaussian family the dispersion is estimated from the residual deviance, and the number of parameters is the rank plus one.
null.deviance	the deviance for the null model, comparable with deviance; the null model includes a non-varying intercept only
iter	number of iterations
weights	working weights of the final iteration
df.residual	the residual degrees of freedom/degrees of freedom error; computed like rank
df.null	the residual degrees of freedom for the null model
converged	logical; fulfills the PIRLS-algorithm the given convergence conditions?
boundary	logical; is the fitted value on the boundary of the attainable values?
offset	the offset vector used
control	the value of the control argument used
contrasts	the contrasts used
na.action	information returned by model.frame on the special handling of NAs; currently always na.omit
plot	in principle, a list containing two matrixes needed for different types of plots: if input option plot=TRUE, the first matrix contains estimates needed to plot coefficient paths; if lambda was cross-validated, the second matrix contains the cross-validation scores
tuning	a list, employed tuning parameters; if lambda was cross-validated, the optimal value is returned
indices	used index argument; see function index
number.selectable.parameters	number of coefficients that could be selected
number.removed.parameters	number of actual removed coefficients
x.reduction	a matrix; transforms model frame x into its reduced version; e.g. needed for refitting
beta.reduction	a matrix; transforms the coefficients into its reduced version
call	the matched call
formula	the formula supplied
terms	the terms object used
data	the data argument
x, y	if requested, the model matrix/the response vector
model	if requested, the model frame
xlevels	a record of the levels of the factors used in fitting
bootstrap.errors	experimental
method	same as input argument method

In addition, non-empty fits will have components qr, R and effects relating to the final weighted linear fit.

Note

Pleas note that the functions gvcm.cat, pest and the fitting procedure for penalized estimation gvcmcatfit are organized like the functions glm/glm.fit whenever possible. This was done to avoid mistakes and to provide a well-known structure.

Author(s)

Margret-Ruth Oelker ([e-mail])

References

Eilers, P. H. C. and B. D. Marx (1996). Flexible smoothing with b-splines and penalties. Statist. Sci. 11 (2), 89-121.
Fan, J. and R. Li (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96(456), 1348-1360.
Gertheiss, J. and G. Tutz (2010). Sparse modeling of categorial explanatory variables. The Annals of Statistics 4(4), 2150-2180.
Oelker, M.-R., J. Gertheiss and G. Tutz (2012). Regularization and model melection with categorial predictors and effect modifiers in generalized linear models. Department of Statistics at the University of Munich: Technical Report 122.
Oelker, M.-R., J. Gertheiss and G. Tutz (2013). A general family of penalties for combining differing types of penalties in generalized structured models. Department of Statistics at the University of Munich: Technical Report 139.
Yuan, M. and Y. Lin (2006). Model selection and estimation in regression with grouped variables. R. Stat. Soc. Ser. B Stat. Methodol. 68 (1), 49-67.
Zou, H. and T. Hastie (2005). Regularization and variable selection via the Elastic Net. R. Stat. Soc. Ser. B Stat. Methodol. 67 (2), 301-320.

See Also

Functions index, cat_control, plot.gvcm.cat, predict.gvcm.cat, simulation

Examples

## example for function simulation()
covariates <- list(x1=list("unif", c(0,2)),
                  x2=list("unif", c(0,2)),
                  x3=list("unif", c(0,2)),
                  u=list("multinom",c(0.3,0.4,0.3), "nominal")
                  )
true.f <- y ~ 1 + v(x1,u) + x2
true.coefs <- c(0.2,  0.3,.7,.7, -.5)
data <- simulation(400, covariates, NULL, true.f, true.coefs , binomial(), seed=456)
## example for function gvcm.cat()
f <- y ~ v(1,u) + v(x1,u) + v(x2,u)
m1 <- gvcm.cat(f, data, binomial(), plot=TRUE, control=cat_control(lambda.upper=19))
summary(m1)
## example for function predict.gvcm.cat
newdata <- simulation(200, covariates, NULL, true.f, true.coefs , binomial(), seed=789)
prediction <- predict.gvcm.cat(m1, newdata) 
## example for function plot.gvcm.cat 
plot(m1)
plot(m1, type="score")
plot(m1, type="coefs")

gvcm.cat documentation built on May 1, 2019, 10:13 p.m.