penMlogistic: penMlogistic

In gonzalochebi/penMlogistic: What the package does (short line)

Description

This function computes a penalized and robust estimator for the Logistic Regression Model.

Usage

penMlogistic(loss = c("ch", "deviance", "divergence", "ls"), penalty = c("sign","en","scad","mcp"), X, Y, use.weights = F, alpha = NA, estimate.intercept = T, lambda.grid = NULL, nlambda = 12, use.importance.sets = T, lambda.grid.eps = 0.2, nfolds = 10, verbose = F, algorithm.method = c("cd", "hjk"))

Arguments

loss	The loss function to be used (see Details for futher information). Defaults to "ch".
penalty	The penalty function to be used (see Details for futher information). Defaults to "sign".
X	The design matrix. Columns should be standarized for better algorithmical stability.
Y	The response vector. Each coordinate must be either 0 or 1.
use.weights	If TRUE, the method uses the hard weights described in Bianco et al. (2019). Defaults to FALSE.
alpha	An additional tuning parameter used for certain penalization functions.
estimate.intercept	If TRUE, the method also estimates an intercept parameter. Defaults to TRUE.
lambda.grid	A vector of candidates for the tuning parameter lambda. Robust Cross-Validation will be performed over this grid to select the optimal value for lambda. If no value is provided, the grid is chosen as described in Bianco et al. (2019).
nlambda	The number of elements of the lambda grid to be generated (only if no value is provided for lambda.grid). Defaults to 12.
use.importance.sets	If TRUE, it implements an heuristic version of the Strong Safe Rules (Tibshirani et al. (2012)) as described in Bianco et al. (2019). Used to improve run time. Defaults to TRUE.
lambda.grid.eps	The ratio between the smallest and largest values of the lambda grid. Defaults to 0.2.
nfolds	The number of Cross Validation folds to be used. Defaults to 10.
algorithm.method	The algorithm to be used (see Details for futher information). Defaults to "cd".

Details

The loss function must be one value from the vector ("ch", "deviance", "divergence", "ls"). The "ch" function corresponds to the one defined in Croux and Haesbroeck (2003), which guarantees the existence of the unpenalized estimator under overlapping conditions. The "deviance" loss is the classic unbounded function used in the maximum likelihood estimator. The "divergence" loss function is the one defined in Basu et al. (1998). Finally, the "ls" loss functions corresponds to the least squares estimator used in Chi and Scott (2014).

The penalty function must be one value from the vector ("sign","en","scad","mcp"). The "sign" penalty is the one defined in Bianco et al. (2019). Moreover, "en" corresponds to the elastic net function defined in Hastie and Zou (2005) and the additional tuning parameter alpha must be defined (and must lie between 0 and 1). When alpha = 1, this corresponds to the Lasso penalty and when alpha = 0, it corresponds to the Ridge penalization. On the other hand, "scad" is the penalty defined in Fan and Li (2001) and the additional tuning parameter alpha must be supplied and greater than 2. Finally, the "mcp" penalty is the one defined in Zhang (2010). In this case, the tuning parameter alpha should also be supplied and must be greater than 1.

The algorithm method must be one value from the vector ("cd","hjk"). The "cd" algorithm corresponds to the implementation of the cyclical descent described in Bianco et al. (2019). The "hjk" algorithm uses the Hooke-Jeeves derivative-free minimization algorithm (from the dfoptim package).

Value

A list with the following elements:

all.lambdas

The used lambda grid

all.betas

The matrix whose rows are the estimated coefficient vectors for each value of the lambda grid

all.betas0

The estimated intercept for each value of the lambda grid

all.df

The number of estimated nonzero coefficients for each value of the lambda grid

beta

The estimated coefficient vector (beta) for the chosen value of the lambda grid

beta0

The estimated intercept (beta0) for the chosen value of the lambda grid

lambda

The chosen value of the lambda grid

References

Tibshirani, R., Bien, J., Friedman, J., Hastie, T., Simon, N., Taylor, J., & Tibshirani, R. J. (2012). Strong rules for discarding predictors in lasso-type problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266.

Chi, E. C., & Scott, D. W. (2014). Robust parametric classification and variable selection by a minimum distance criterion. Journal of Computational and Graphical Statistics, 23, 111-128.

Croux, C., & Haesbroeck, G. (2003). Implementing the Bianco and Yohai estimator for logistic regression. Computational statistics \& data analysis, 44, 273-295.

Fan, J., & Li, R. (2001). Variable selection via non–concave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348-1360.

Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology, 67(2), 301-320.

Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of statistics, 38, 894-942.

Basu, A., Gosh, A., Mandal, A., Martin, N. & Pardo, L. (2017). A Wald–type test statistic for testing linear hypothesis in logistic regression models based on minimum density power divergence estimator. Electronic Journal of Statistics, 11, 2741-2772.

Examples

X = scale(matrix(rnorm(2000), 200, 10))
beta = c(rep(1,3), rep(0,7))
probabilities = 1 / (1 + exp(-X %*% beta))
Y = rbinom(n = 200, size = 1, prob = probabilities)
penMlogistic(loss = "divergence", penalty = "sign", X = X, Y = Y) 

gonzalochebi/penMlogistic documentation built on Nov. 4, 2019, 1:20 p.m.