Demonstration of overfitting
This function gives a demonstration of how overfitting occurs on a user-inputted dataset by showing the estimated generalization error as additional variables are added to the regression model (up to all two-way interactions).
The data frame where demonstration will occur.
The response variable (in quotes)
Optional argument setting the random number seed if results need to be reproduced
This function splits
DF in half to obtain training and holdout samples. Regression models are constructed using a forward selection procedure (adding the variable that decreases the AIC the most on the training set), starting at the naive model and terminating at the full model with all two-way interactions.
The generalization error of each model is computed on the holdout sample. The AIC and generalization errors are plotted versus the number of variables in the model to illustrate overfitting. Typically, the generalization error decreases at first as useful variables are added to the model, then the generalization error increases after the new variables added start to fit the quirks present only in the training data. When this happens, the model is said to be overfit.
Introduction to Regression and Modeling
1 2 3 4 5 6 7 8
Want to suggest features or report bugs for rdrr.io? Use the GitHub issue tracker.