Variable Selection Methods"

In olsrr: Tools for Building OLS Regression Models

library(olsrr)
library(ggplot2)
library(gridExtra)
library(nortest)
library(goftest)

Introduction

All Possible Regression

All subset regression tests all possible subsets of the set of potential independent variables. If there are K potential independent variables (besides the constant), then there are $2^{k}$ distinct subsets of them to be tested. For example, if you have 10 candidate independent variables, the number of subsets to be tested is $2^{10}$, which is 1024, and if you have 20 candidate variables, the number is $2^{20}$, which is more than one million.

model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
ols_step_all_possible(model)

The plot method shows the panel of fit criteria for all possible regression methods.

model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
k <- ols_step_all_possible(model)
plot(k)

Best Subset Regression

Select the subset of predictors that do the best at meeting some well-defined objective criterion, such as having the largest R2 value or the smallest MSE, Mallow's Cp or AIC.

model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
ols_step_best_subset(model)

The plot method shows the panel of fit criteria for best subset regression methods.

model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
k <- ols_step_best_subset(model)
plot(k)

Stepwise Forward Regression

Build regression model from a set of candidate predictor variables by entering predictors based on p values, in a stepwise manner until there is no variable left to enter any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_p(model)

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_forward_p(model)
plot(k)

Detailed Output

# stepwise forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_p(model, details = TRUE)

Stepwise Backward Regression

Build regression model from a set of candidate predictor variables by removing predictors based on p values, in a stepwise manner until there is no variable left to remove any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_p(model)

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_backward_p(model)
plot(k)

Detailed Output

# stepwise backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_p(model, details = TRUE)

Stepwise Regression

Build regression model from a set of candidate predictor variables by entering and removing predictors based on p values, in a stepwise manner until there is no variable left to enter or remove any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise regression
model <- lm(y ~ ., data = surgical)
ols_step_both_p(model)

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_both_p(model)
plot(k)

Detailed Output

# stepwise regression
model <- lm(y ~ ., data = surgical)
ols_step_both_p(model, details = TRUE)

Stepwise AIC Forward Regression

Build regression model from a set of candidate predictor variables by entering predictors based on Akaike Information Criteria, in a stepwise manner until there is no variable left to enter any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise aic forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_aic(model)

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_forward_aic(model)
plot(k)

Detailed Output

# stepwise aic forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_aic(model, details = TRUE)

Stepwise AIC Backward Regression

Build regression model from a set of candidate predictor variables by removing predictors based on Akaike Information Criteria, in a stepwise manner until there is no variable left to remove any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise aic backward regression
model <- lm(y ~ ., data = surgical)
k <- ols_step_backward_aic(model)
k

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_backward_aic(model)
plot(k)

Detailed Output

# stepwise aic backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_aic(model, details = TRUE)

Stepwise AIC Regression

Build regression model from a set of candidate predictor variables by entering and removing predictors based on Akaike Information Criteria, in a stepwise manner until there is no variable left to enter or remove any more. The model should include all the candidate predictor variables. If details is set to TRUE, each step is displayed.

Variable Selection

# stepwise aic regression
model <- lm(y ~ ., data = surgical)
ols_step_both_aic(model)

Plot

model <- lm(y ~ ., data = surgical)
k <- ols_step_both_aic(model)
plot(k)

Detailed Output

# stepwise aic regression
model <- lm(y ~ ., data = surgical)
ols_step_both_aic(model, details = TRUE)

olsrr documentation built on Feb. 10, 2020, 5:07 p.m.