msgl-package: Multinomial logistic regression with sparse group lasso...

In nielsrhansen/msgl: Multinomial Sparse Group Lasso

Multinomial logistic regression with sparse group lasso penalty.

Description

Simultaneous feature selection and parameter estimation for classification. Suitable for high dimensional multiclass classification with many classes. The algorithm computes the sparse group lasso penalized maximum likelihood estimate. Use of parallel computing for cross validation and subsampling is supported through the foreach and doParallel packages. Development version is on GitHub, please report package issues on GitHub.

Details

For a classification problem with K classes and p features (covariates) dived into m groups. The multinomial logistic regression with sparse group lasso penalty estimator is a sequence of minimizers (one for each lambda given in the lambda argument) of

\hat R(\beta) + \lambda \left( (1-\alpha) \sum_{J=1}^m \gamma_J \|\beta^{(J)}\|_2 + \alpha \sum_{i=1}^{n} \xi_i |\beta_i| \right)

where \hat R is the weighted empirical log-likelihood risk of the multinomial regression model. The vector \beta^{(J)} denotes the parameters associated with the J'th group of features (default is one covariate per group, hence the default dimension of \beta^{(J)} is K). The group weights \gamma \in [0,\infty)^m and parameter weights \xi \in [0,\infty)^n may be explicitly specified.

Author(s)

Martin Vincent

Examples

# Load some data
data(PrimaryCancers)

# A quick look at the data
dim(x)
table(classes)

# A smaller subset with three classes 
small <- which(classes %in% c("CCA", "CRC", "Pancreas"))
classes <- classes[small, drop = TRUE]
x <- x[small, ]

#Do cross validation using 2 parallel units
cl <- makeCluster(2)
registerDoParallel(cl)

# Do 4-fold cross validation on a lambda sequence of length 100.
# The sequence is decreasing from the data derived lambda.max to 0.2*lambda.max
fit.cv <- msgl::cv(x, classes, fold = 4, lambda = 0.2, use_parallel = TRUE)

stopCluster(cl)

# Print information about models
# and cross validation errors (estimated expected generalization error)
fit.cv

nielsrhansen/msgl documentation built on Feb. 6, 2024, 1:25 a.m.