mldr_evaluate: Evaluate predictions made by a multilabel classifier

In mldr: Exploratory Data Analysis and Manipulation of Multi-Label Data Sets

Description

Taking as input an mldr object and a matrix with the predictions given by a classifier, this function evaluates the classifier performance through several multilabel metrics.

Usage

mldr_evaluate(mldr, predictions, threshold = 0.5)

Arguments

mldr	Object of "mldr" class containing the instances to evaluate
predictions	Matrix with the labels predicted for each instance in the mldr parameter. Each element should be a value into [0,1] range
threshold	Threshold to use to generate bipartition of labels. By default the value 0.5 is used

Value

A list with multilabel predictive performance measures. The items in the list will be

• accuracy

• example_auc

• average_precision

• coverage

• fmeasure

• hamming_loss

• macro_auc

• macro_fmeasure

• macro_precision

• macro_recall

• micro_auc

• micro_fmeasure

• micro_precision

• micro_recall

• one_error

• precision

• ranking_loss

• recall

• subset_accuracy

• roc

The roc element corresponds to a roc object associated to the MicroAUC value. This object can be given as input to plot for plotting the ROC curve The example_auc, macro_auc, micro_auc and roc members will be NULL if the pROC package is not installed.

See Also

mldr, Basic metrics, Averaged metrics, Ranking-based metrics, roc.mldr

Examples

## Not run: 
library(mldr)

# Get the true labels in emotions
predictions <- as.matrix(emotions$dataset[, emotions$labels$index])
# and introduce some noise (alternatively get the predictions from some classifier)
noised_labels <- cbind(sample(1:593, 200, replace = TRUE), sample(1:6, 200, replace = TRUE))
predictions[noised_labels] <- sample(0:1, 100, replace = TRUE)
# then evaluate predictive performance
res <- mldr_evaluate(emotions, predictions)
str(res)
plot(res$roc, main = "ROC curve for emotions")

## End(Not run)

Example output

List of 20
 $ accuracy         : num 0.935
 $ example_auc      : num 0.974
 $ average_precision: num 0.964
 $ coverage         : num 1.03
 $ fmeasure         : num 0.957
 $ hamming_loss     : num 0.0275
 $ macro_auc        : num 0.975
 $ macro_fmeasure   : num 0.955
 $ macro_precision  : num 0.932
 $ macro_recall     : num 0.979
 $ micro_auc        : num 0.974
 $ micro_fmeasure   : num 0.957
 $ micro_precision  : num 0.936
 $ micro_recall     : num 0.978
 $ one_error        : num 0.0438
 $ precision        : num 0.951
 $ ranking_loss     : num 0.0508
 $ recall           : num 0.979
 $ subset_accuracy  : num 0.841
 $ roc              :List of 15
  ..$ percent           : logi FALSE
  ..$ sensitivities     : num [1:3] 1 0.978 0
  ..$ specificities     : num [1:3] 0 0.97 1
  ..$ thresholds        : num [1:3] -Inf 0.5 Inf
  ..$ direction         : chr "<"
  ..$ cases             : num [1:1108] 1 1 1 1 1 1 1 1 1 0 ...
  ..$ controls          : num [1:2450] 1 0 0 0 0 0 0 0 0 0 ...
  ..$ fun.sesp          :function (thresholds, controls, cases, direction)  
  ..$ auc               :Classes 'auc', 'numeric'  atomic [1:1] 0.974
  .. .. ..- attr(*, "partial.auc")= logi FALSE
  .. .. ..- attr(*, "percent")= logi FALSE
  .. .. ..- attr(*, "roc")=List of 15
  .. .. .. ..$ percent           : logi FALSE
  .. .. .. ..$ sensitivities     : num [1:3] 1 0.978 0
  .. .. .. ..$ specificities     : num [1:3] 0 0.97 1
  .. .. .. ..$ thresholds        : num [1:3] -Inf 0.5 Inf
  .. .. .. ..$ direction         : chr "<"
  .. .. .. ..$ cases             : num [1:1108] 1 1 1 1 1 1 1 1 1 0 ...
  .. .. .. ..$ controls          : num [1:2450] 1 0 0 0 0 0 0 0 0 0 ...
  .. .. .. ..$ fun.sesp          :function (thresholds, controls, cases, direction)  
  .. .. .. ..$ auc               :Classes 'auc', 'numeric'  atomic [1:1] 0.974
  .. .. .. .. .. ..- attr(*, "partial.auc")= logi FALSE
  .. .. .. .. .. ..- attr(*, "percent")= logi FALSE
  .. .. .. .. .. ..- attr(*, "roc")=List of 8
  .. .. .. .. .. .. ..$ percent      : logi FALSE
  .. .. .. .. .. .. ..$ sensitivities: num [1:3] 1 0.978 0
  .. .. .. .. .. .. ..$ specificities: num [1:3] 0 0.97 1
  .. .. .. .. .. .. ..$ thresholds   : num [1:3] -Inf 0.5 Inf
  .. .. .. .. .. .. ..$ direction    : chr "<"
  .. .. .. .. .. .. ..$ cases        : num [1:1108] 1 1 1 1 1 1 1 1 1 0 ...
  .. .. .. .. .. .. ..$ controls     : num [1:2450] 1 0 0 0 0 0 0 0 0 0 ...
  .. .. .. .. .. .. ..$ fun.sesp     :function (thresholds, controls, cases, direction)  
  .. .. .. .. .. .. ..- attr(*, "class")= chr "roc"
  .. .. .. ..$ call              : language roc.default(response = as.integer(mldr_to_labels(mldr)), predictor = as.numeric(predictions),      algorithm = 3)
  .. .. .. ..$ original.predictor: num [1:3558] 1 1 0 0 0 0 1 0 1 0 ...
  .. .. .. ..$ original.response : int [1:3558] 0 1 0 0 0 0 1 0 1 0 ...
  .. .. .. ..$ predictor         : num [1:3558] 1 1 0 0 0 0 1 0 1 0 ...
  .. .. .. ..$ response          : int [1:3558] 0 1 0 0 0 0 1 0 1 0 ...
  .. .. .. ..$ levels            : chr [1:2] "0" "1"
  .. .. .. ..- attr(*, "class")= chr "roc"
  ..$ call              : language roc.default(response = as.integer(mldr_to_labels(mldr)), predictor = as.numeric(predictions),      algorithm = 3)
  ..$ original.predictor: num [1:3558] 1 1 0 0 0 0 1 0 1 0 ...
  ..$ original.response : int [1:3558] 0 1 0 0 0 0 1 0 1 0 ...
  ..$ predictor         : num [1:3558] 1 1 0 0 0 0 1 0 1 0 ...
  ..$ response          : int [1:3558] 0 1 0 0 0 0 1 0 1 0 ...
  ..$ levels            : chr [1:2] "0" "1"
  ..- attr(*, "class")= chr "roc"

mldr documentation built on Jan. 11, 2020, 9:18 a.m.