backward: Backward Selection Algorithm

In UBC-MDS/punisheR: Feature Selection Using AIC and BIC

Description

This is an implementation of the backward selection algorithm in which you start with a full model and iteratively remove the least useful feature at each step. This function is built for the specific case of backward selection in linear regression models.

Usage

backward(X_train, y_train, X_val, y_val, n_features = 0.5,
  min_change = NULL, criterion = "r-squared", verbose = TRUE)

Arguments

X_train	Training data. Represented as a 2D matrix or dataframe of (observations, features).
y_train	Target class for training data. Represented as a 1D vector of target classes for X_train. If y_train is a character string AND X is a dataframe, it will be extracted from X.
X_val	Validation data. Represented as a 2D matrix or dataframe of (observations, features).
y_val	Target class for validation data. Represented as a 1D vector of target classes for X_val. If y_val is a character string AND X is a dataframe, it will be extracted from X.
n_features	The number of features to select, expressed either as a proportion (0,1) or whole number with range (0,total_features). Note: min_change must be NULL if this is numeric.
min_change	The smallest change in criterion score to be considered significant. Note: n_features must be NULL if this is numeric.
criterion	Model selection criterion to measure relative model quality. Can be one of: 'aic': use Akaike Information Criterion 'bic': use Akaike Information Criterion 'r-squared': use coefficient of determination
verbose	if TRUE, print additional information as selection occurs

Value

A vector of indices that represent the best features of the model.

Examples

X_train <- matrix(runif(50, 0, 50), ncol=5)
y_train <- runif(10, 0, 50)
X_val <- matrix(runif(50, 0, 20), ncol=5)
y_val <- runif(10, 0, 20)
backward(X_train, y_train, X_val, y_train, min_change=0.1, n_features=NULL, criterion="r-squared")
backward(X_train, y_train, X_val, y_train, n_features=0.1, min_change=NULL, criterion="aic")

UBC-MDS/punisheR documentation built on May 25, 2019, 1:36 p.m.