penalreg: Correlation-based Penalty
In lqa: Penalized Likelihood Inference for GLMs

Description Usage Arguments Details Value Author(s) References See Also

Object of the penalty to handle the correlation-based penalty (Tutz \& Ulbricht, 2009).

1	penalreg(lambda = NULL, ...)

`lambda`	regularization parameter. This must be a nonnegative real number.
`...`	further arguments

The method proposed in Tutz \& Ulbricht (2009) and Ulbricht \& Tutz (2008) utilizes the correlation between regressors explicitly in the penalty term. Coefficients which correspond to pairs of covariates are weighted according to their marginal correlation. The correlation-based penalty is given by

P_{λ}^{cb}(\boldsymbol{β}) = \frac{λ}{2} ∑_{i=1}^{p-1}∑_{j > i}≤ft\{ \frac{(β_{i}-β_{j})^{2}}{1-\varrho_{ij}} + \frac{(β_{i}+β_{j})^{2}}{1+\varrho_{ij}}\right\}

where \varrho_{ij} denotes the (empirical) correlation between the i-th and the j-th regressor. It is designed in a way so that for strong positive correlation (\varrho_{ij}\uparrow 1) the first term becomes dominant having the effect that estimates for β_i and β_j are similar (\hatβ_i\approx\hatβ_j). For strong negative correlation (\varrho_{ij}\downarrow -1) the second term becomes dominant and \hatβ_i will be close to -\hatβ_j. The effect is grouping, highly correlated effects show comparable values of estimates (|\hatβ_i|\approx|\hatβ_j|) with the sign being determined by positive or negative correlation. If the regressors are uncorrelated (\varrho_{ij}=0) one obtains (up to a constant) the ridge penalty, i.e. P_λ^{cb}(\boldsymbol{β})\proptoλ∑_{i=1}^pβ_i^2. Consequently, for weakly correlated data the performance is quite close to the ridge estimator. Therefore, as for the elastic net ridge regression is a limiting case.

The correlation-based penalty is a quadratic penalty. Consequently, in general it will not be able to select variables. For this reason there have been introduced some advanced boosting techniques, such as GBlockBoost or ForwardBoost. See GBlockBoost and ForwardBoost for further details.

An object of the class penalty. This is a list with elements

`penalty`	character: the penalty name.
`lambda`	double: the (nonnegative) regularization parameter.
`getpenmat`	function: computes the diagonal penalty matrix.

Jan Ulbricht

Tutz, G. \& J. Ulbricht (2009) Penalized Regression with correlation based penalty. Statistics and Computing 19, 239–253.

Ulbricht, J. \& G. Tutz (2008) Boosting correlation based penalization in generalized linear models. In Shalabh \& C. Heumann (Eds.) Recent Advances in Linear Models and Related Areas. Heidelberg: Springer.

penalty, ridge, lasso, GBlockBoost, ForwardBoost