multi_pr: Multi-class classification PR

In multiROC: Calculating and Visualizing ROC and PR Curves Across Multi-Class Classifications

Description

This function calculates the Precision, Recall and AUC of multi-class classifications.

Usage

multi_pr(data, force_diag=TRUE)

Arguments

data	A data frame contain true labels of multiple groups and corresponding predictive scores
force_diag	If TRUE, TPR and FPR will be forced to across (0, 0) and (1, 1)

Details

A data frame is required for this function as input. This data frame should contains true label (0 - Negative, 1 - Positive) columns named as XX_true (e.g. S1_true, S2_true and S3_true) and predictive scores (continuous) columns named as XX_pred_YY (e.g. S1_pred_SVM, S2_pred_RF), thus this function allows calcluating ROC on mulitiple classifiers.

Predictive scores could be probabilities among [0, 1] and other continuous values. For each classifier, the number of columns should be equal to the number of groups of true labels. The order of columns won't affect results.

Recall, Precision, AUC for each group and each method will be calculated. Macro/Micro-average AUC for all groups and each method will be calculated.

Micro-average ROC/AUC was calculated by stacking all groups together, thus converting the multi-class classification into binary classification. Macro-average ROC/AUC was calculated by averaging all groups results (one vs rest) and linear interpolation was used between points of ROC.

AUC will be calculated using function cal_auc().

Value

Recall	A list of recalls for each group, each method and micro-/macro- average
Precision	A list of precisions for each group, each method and micro-/macro- average
AUC	A list of AUCs for each group, each method and micro-/macro- average
Methods	A vector contains the name of different classifiers
Groups	A vector contains the name of different groups

Examples

data(test_data)
pr_test <- multi_pr(test_data)
pr_test$AUC 

Example output

Warning messages:
1: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
2: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
3: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
4: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
5: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
6: In regularize.values(x, y, ties, missing(ties), na.rm = na.rm) :
  collapsing to unique 'x' values
$m1
$m1$G1
[1] 0.8110409

$m1$G2
[1] 0.188981

$m1$G3
[1] 0.6747914

$m1$macro
[1] 0.5496887

$m1$micro
[1] 0.7512521


$m2
$m2$G1
[1] 0.6063347

$m2$G2
[1] 0.1329879

$m2$G3
[1] 0.08150105

$m2$macro
[1] 0.2732088

$m2$micro
[1] 0.2754047

multiROC documentation built on May 1, 2019, 10:11 p.m.