confMatrixMetrics: Obtain several metrics based on the confusion matrix

In CustomerScoringMetrics: Evaluation Metrics for Customer Scoring Models Depending on Binary Classifiers

Description

Calculates a range of metrics based upon the confusion matrix: accuracy, true positive rate (TPR; sensitivity or recall), true negative rate (specificity), false postive rate (FPR), false negative rate (FPR), F1-score , with the optional ability to dynamically determine an incidence-based cutoff value using validation sample predictions.

Usage

confMatrixMetrics(predTest, depTest, cutoff = 0.5, dyn.cutoff = FALSE,
  predVal = NULL, depVal = NULL)

Arguments

predTest	Vector with predictions (real-valued or discrete)
depTest	Vector with real class labels
cutoff	Threshold for converting real-valued predictions into class predictions. Default 0.5.
dyn.cutoff	Logical indicator to enable dynamic threshold determination using validation sample predictions. In this case, the function determines, using validation data, the incidence (occurrence percentage of the customer behavior or characterstic of interest) and chooses a cutoff value so that the number of predicted positives is equal to the number of true positives. If TRUE, then the value for the cutoff parameter is ignored.
predVal	Vector with predictions (real-valued or discrete). Only used if dyn.cutoff is TRUE.
depVal	Optional vector with true class labels for validation data. Only used if dyn.cutoff is TRUE.

Value

A list with the following items:

accuracy	accuracy value
truePostiveRate	TPR or true positive rate
trueNegativeRate	TNR or true negative rate
falsePostiveRate	FPR or false positive rate
falseNegativeRate	FNR or false negative rate
F1Score	F1-score
cutoff	the threshold value used to convert real-valued predictions to class predictions

Author(s)

Koen W. De Bock, [e-mail]

References

Witten, I.H., Frank, E. (2005): Data Mining: Practical Machine Learning Tools and Techniques, Second Edition. Chapter 5. Morgan Kauffman.

See Also

dynConfMatrix,dynAccuracy

Examples

## Load response modeling data set
data("response")
## Apply confMatrixMetrics function to obtain confusion matrix-based performance metrics
## achieved on the test sample. Use validation sample predictions to dynamically
## determine a cutoff value.
cmm<-confMatrixMetrics(response$test[,2],response$test[,1],dyn.cutoff=TRUE,
predVal=response$val[,2],depVal=response$val[,1])
## Retrieve F1-score
print(cmm$F1Score)

CustomerScoringMetrics documentation built on May 2, 2019, 5:17 a.m.