In m-clark/confusionMatrix: Confusion Matrix

knitr::opts_chunk$set(
  fig.path = "man/figures/",
  comment = NA
)

Basic Usage

Two class

set.seed(1234)

N = 2500
x = rnorm(N)
lp = 0 + .5*x
y = rbinom(N, size = 1, prob = plogis(lp))

model = glm(y ~ x, family = binomial)
predict_class = predict(model) > 0

library(confusionMatrix)
confusion_matrix(predict_class, y)

Multi-class

Works for the multi-class setting too, but not all statistics are applicable/available. Note that the 'Other' statistics are provided for each class, as well as averaged.

p_multi = sample(letters[1:4], 250, replace = TRUE, prob = 1:4)
o_multi = sample(letters[1:4], 250, replace = TRUE, prob = 1:4)

confusion_matrix(p_multi, o_multi)

Available Statistics

Accuracy

• Accuracy
• Lower and Upper Limits
• Guessing rate
• Test of accuracy vs. the guessing rate

Additional Statistics

• Relative class sizes
• Sensitivity/Recall/True Positive Rate
• Specificity/True Negative Rate
• Positive Predictive Value/Precision
• Negative Predictive Value
• F1 score/Dice coefficient
• Prevalence
• Detection Rate
• Detection Prevalence
• Balanced Accuracy
• False Discovery Rate
• False Omission Rate
• False Positive Rate/Fallout
• False Negative Rate
• D'
• Area Under a ROC

Association and Agreement

• Kappa
• Adjusted Rand
• Yule
• Phi
• Peirce/Youden J
• Jaccard

Long vs. Wide

In wide format, you have a simple way to pluck any particular statistic.

cm = confusion_matrix(predict_class, y)

cm[['Other']][['N']]
cm[['Association and Agreement']][['Yule']]

However, if you want to present a lot of statistics, a longer format might be more ideal.

confusion_matrix(predict_class, y, longer = TRUE)

Either way, the tibble output will work nicely with various packages for formatting tables.

m-clark/confusionMatrix documentation built on July 15, 2020, 4:16 p.m.