smbinning.metrics: Performance Metrics for a Classification Model

In smbinning: Scoring Modeling and Optimal Binning

Description

It computes the classic performance metrics of a scoring model, including AUC, KS and all the relevant ones from the classification matrix at a specific threshold or cutoff.

Usage

smbinning.metrics(dataset, prediction, actualclass, cutoff = NA,
  report = 1, plot = "none", returndf = 0)

Arguments

dataset	Data frame.
prediction	Classifier. A value generated by a classification model (Must be numeric).
actualclass	Binary variable (0/1) that represents the actual class (Must be numeric).
cutoff	Point at wich the classifier splits (predicts) the actual class (Must be numeric). If not specified, it will be estimated by using the maximum value of Youden J (Sensitivity+Specificity-1). If not found in the data frame, it will take the closest lower value.
report	Indicator defined by user. 1: Show report (Default), 0: Do not show report.
plot	Specifies the plot to be shown for overall evaluation. It has three options: 'auc' shows the ROC curve, 'ks' shows the cumulative distribution of the actual class and its maximum difference (KS Statistic), and 'none' (Default).
returndf	Option for the user to save the data frame behind the metrics. 1: Show data frame, 0: Do not show (Default).

Value

The command smbinning.metrics returns a report with classic performance metrics of a classification model.

Examples

# Load library and its dataset
library(smbinning) # Load package and its data

# Example: Metrics Credit Score 1
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
                  report=1) # Show report
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
                  cutoff=600, report=1) # User cutoff
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
                  report=0, plot="auc") # Plot AUC
smbinning.metrics(dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
                  report=0, plot="ks") # Plot KS

# Save table with all details of metrics
cbs1metrics=smbinning.metrics(
  dataset=smbsimdf1,prediction="cbs1",actualclass="fgood",
  report=0, returndf=1) # Save metrics details

Example output

Loading required package: sqldf
Loading required package: gsubfn
Loading required package: proto
Loading required package: RSQLite
Loading required package: partykit
Loading required package: grid
Loading required package: libcoin
Loading required package: mvtnorm
Loading required package: Formula
Warning message:
no DISPLAY variable so Tk is not available 

  Overall Performance Metrics 
  -------------------------------------------------- 
                    KS : 0.3010 (Fair)
                   AUC : 0.6990 (Poor)

  Classification Matrix 
  -------------------------------------------------- 
           Cutoff (>=) : 51.78 (Optimal)
   True Positives (TP) : 1044
  False Positives (FP) : 126
  False Negatives (FN) : 751
   True Negatives (TN) : 323
   Total Positives (P) : 1795
   Total Negatives (N) : 449

  Business/Performance Metrics 
  -------------------------------------------------- 
      %Records>=Cutoff : 0.5214
             Good Rate : 0.8923 (Vs 0.7999 Overall)
              Bad Rate : 0.1077 (Vs 0.2001 Overall)
        Accuracy (ACC) : 0.6092
     Sensitivity (TPR) : 0.5816
 False Neg. Rate (FNR) : 0.4184
 False Pos. Rate (FPR) : 0.2806
     Specificity (TNR) : 0.7194
       Precision (PPV) : 0.8923
  False Discovery Rate : 0.1077
    False Omision Rate : 0.6993
  Inv. Precision (NPV) : 0.3007

  Note: 256 rows deleted due to missing data.

[1] "'cutoff' out of range."

smbinning documentation built on May 1, 2019, 10:06 p.m.