TModC: Compare Classification Models

In ModTools: Tools for Building Regression and Classification Models

Description

For the comparison of several classification models, the AUC values and BrierScore values of the models are determined and tabulated. Both the absolute values and the relative values are reported, each related to the model with the highest corresponding value.

Usage

TModC(..., newdata = NULL, reference = NULL, ord = NULL)

## S3 method for class 'TModC'
plot(x, col = NULL, args.legend = NULL,...)

Arguments

...	the models to be compared
x	TModC object to plot
newdata	the data to use for predicting. If not provided, the model.frame will be used.
reference	the reference values
ord	character defining the order of the results table, can be any of "auc", "bs", "auc_p", "bs_p", "bs_rnk", "auc_rnk", "ensemble" (using the mean of "auc_p" and "bs_p" for the ranking).
col	the color for the lines in the ROC plot
args.legend	the legend to be placed in the ROC plot

Value

a matrix with the columns

auc	absolute value of area under the ROC curve (AUC)
auc_p	percentage of the auc based on the best observerd AUC
auc_rnk	the rank of the auc
bs	absolute value of the Brier score
bs_p	percentage of the Brier score based on the best observed BS
bs_rnk	the rank of the BS
auc_grnk	character representation of the AUC rank
bs_grnk	character representation of the BS rank

Author(s)

Andri Signorell <[e-mail]>

See Also

TMod, BrierScore, AUC, ROC

Examples

d.pim <- SplitTrainTest(d.pima, p = 0.2)
mdiab <- formula(diabetes ~ pregnant + glucose + pressure + triceps +
                            insulin + mass + pedigree + age)

r.glm <- FitMod(mdiab, data=d.pim$train, fitfn="logit")
r.rp <- FitMod(mdiab, data=d.pim$train, fitfn="rpart")
mods <- list(glm=r.glm, rp=r.rp)

# the table with the measures
(tm <- TModC(mods, ord="auc"))

# plotting the ROC curves
plot(tm, col=c("darkmagenta", "dodgerblue"))

ModTools documentation built on July 26, 2021, 9:06 a.m.