forward: Forward Selection Algorithm.

In UBC-MDS/punisheR: Feature Selection Using AIC and BIC

Description

This is an implementation of the forward selection algorithm in which you start with a null model and iteratively add the most useful features. This function is built for the specific case of forward selection in linear regression.

Usage

forward(X_train, y_train, X_val, y_val, min_change = 0.5, n_features = NULL,
  criterion = "r-squared", verbose = TRUE)

Arguments

X_train	Training data. Represented as a 2D matrix or dataframe of (observations, features).
y_train	Target class for training data. Represented as a 1D vector of target classes for X_train. If y_train is a character string AND X is a dataframe, it will be extracted from X.
X_val	Validation data. Represented as a 2D matrix or dataframe of (observations, features).
y_val	Target class for validation data. Represented as a 1D vector of target classes for X_val. If y_val is a character string AND X is a dataframe, it will be extracted from X.
min_change	The smallest change in criterion score to be considered significant. Note: n_features must be NULL if this is numeric.
n_features	The number of features to select, expressed either as a proportion (0,1) or whole number with range (0,total_features). Note: min_change must be NULL if this is numeric.
criterion	Model selection criterion to measure relative model quality. Can be one of: 'aic': use Akaike Information Criterion 'bic': use Akaike Information Criterion 'r-squared': use coefficient of determination
verbose	If TRUE, print additional information as selection occurs

Value

A vector of indices that represent the best features of the model.

Examples

X_train <- matrix(runif(50, 0, 50), ncol=5)
y_train <- runif(10, 0, 50)
X_val <- matrix(runif(50, 0, 20), ncol=5)
y_val <- runif(10, 0, 20)
forward(X_train, y_train, X_val, y_train, min_change=0.1, criterion="r-squared")
forward(X_train, y_train, X_val, y_train, n_features=0.1, criterion="aic")

UBC-MDS/punisheR documentation built on May 25, 2019, 1:36 p.m.