Description Usage Arguments Value Examples
It computes the classic performance metrics of a scoring model, including AUC, KS and all the relevant ones from the classification matrix at a specific threshold or cutoff.
1 2 | smbinning.metrics(dataset, prediction, actualclass, cutoff = NA,
report = 1, plot = "none", returndf = 0)
|
dataset |
Data frame. |
prediction |
Classifier. A value generated by a classification model (Must be numeric). |
actualclass |
Binary variable (0/1) that represents the actual class (Must be numeric). |
cutoff |
Point at wich the classifier splits (predicts) the actual class (Must be numeric). If not specified, it will be estimated by using the maximum value of Youden J (Sensitivity+Specificity-1). If not found in the data frame, it will take the closest lower value. |
report |
Indicator defined by user. 1: Show report (Default), 0: Do not show report. |
plot |
Specifies the plot to be shown for overall evaluation. It has three options: 'auc' shows the ROC curve, 'ks' shows the cumulative distribution of the actual class and its maximum difference (KS Statistic), and 'none' (Default). |
returndf |
Option for the user to save the data frame behind the metrics. 1: Show data frame, 0: Do not show (Default). |
The command smbinning.metrics
returns a report with classic performance metrics of a classification model.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | # Load library and its dataset
library(smbinning) # Load package and its data
# Example: Metrics Credit Score 1
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
report=1) # Show report
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
cutoff=600, report=1) # User cutoff
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
report=0, plot="auc") # Plot AUC
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
report=0, plot="ks") # Plot KS
# Save table with all details of metrics
cbs1metrics=smbinning.metrics(
dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
report=0, returndf=1) # Save metrics details
|
Loading required package: sqldf
Loading required package: gsubfn
Loading required package: proto
Loading required package: RSQLite
Loading required package: partykit
Loading required package: grid
Loading required package: libcoin
Loading required package: mvtnorm
Loading required package: Formula
Warning message:
no DISPLAY variable so Tk is not available
Overall Performance Metrics
--------------------------------------------------
KS : 0.3010 (Fair)
AUC : 0.6990 (Poor)
Classification Matrix
--------------------------------------------------
Cutoff (>=) : 51.78 (Optimal)
True Positives (TP) : 1044
False Positives (FP) : 126
False Negatives (FN) : 751
True Negatives (TN) : 323
Total Positives (P) : 1795
Total Negatives (N) : 449
Business/Performance Metrics
--------------------------------------------------
%Records>=Cutoff : 0.5214
Good Rate : 0.8923 (Vs 0.7999 Overall)
Bad Rate : 0.1077 (Vs 0.2001 Overall)
Accuracy (ACC) : 0.6092
Sensitivity (TPR) : 0.5816
False Neg. Rate (FNR) : 0.4184
False Pos. Rate (FPR) : 0.2806
Specificity (TNR) : 0.7194
Precision (PPV) : 0.8923
False Discovery Rate : 0.1077
False Omision Rate : 0.6993
Inv. Precision (NPV) : 0.3007
Note: 256 rows deleted due to missing data.
[1] "'cutoff' out of range."
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.