kfold.vld: K-fold model cross-validation

In PDtoolkit: Collection of Tools for PD Rating Model Development and Validation

K-fold model cross-validation

Description

kfold.vld performs k-fold model cross-validation. The main goal of this procedure is to generate main model performance metrics such as absolute mean square error, root mean square error or area under curve (AUC) based on resampling method.

Usage

kfold.vld(model, k = 10, seed = 1984)

Arguments

model	Model in use, an object of class inheriting from "glm"
k	Number of folds. If k is equal or greater than the number of observations of modeling data frame, then validation procedure is equivalent to leave one out cross-validation (LOOCV) method. For LOOCV, AUC is not calculated. Default is set to 10.
seed	Random seed needed for ensuring the result reproducibility. Default is 1984.

Value

The command kfold.vld returns a list of two objects.
The first object (iter), returns iteration performance metrics.
The second object (summary), is the data frame of iterations averages of performance metrics.

Examples

suppressMessages(library(PDtoolkit))
data(loans)
#run stepFWD
res <- stepFWD(start.model = Creditability ~ 1, 
	   coding = "WoE",
	   db = loans)
#check output elements
names(res)
#extract the final model
final.model <- res$model
#print coefficients
summary(final.model)$coefficients
#print head of coded development data
head(res$dev.db)
#calculate AUC
auc.model(predictions = predict(final.model, type = "response", newdata = res$dev.db),
    observed = res$dev.db$Creditability)
kfold.vld(model = final.model, k = 10, seed = 1984)

PDtoolkit documentation built on Sept. 20, 2023, 9:06 a.m.