Description Usage Arguments Details Value See Also Examples
Functions that compute ranking-based metrics, given a matrix of true labels and a matrix of predicted probabilities.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | average_precision(true_labels, predictions, ...)
one_error(true_labels, predictions)
coverage(true_labels, predictions, ...)
ranking_loss(true_labels, predictions)
macro_auc(true_labels, predictions, undefined_value = 0.5,
na.rm = FALSE)
micro_auc(true_labels, predictions)
example_auc(true_labels, predictions, undefined_value = 0.5,
na.rm = FALSE)
|
true_labels |
Matrix of true labels, columns corresponding to labels and rows to instances. |
predictions |
Matrix of probabilities predicted by a classifier. |
... |
Additional parameters to be passed to the ranking function. |
undefined_value |
A default value for the cases when macro-averaged
and example-averaged AUC encounter undefined (not computable) values, e.g.
|
na.rm |
Logical specifying whether to ignore undefined values when
|
Available metrics in this category
average_precision
: Example and ranking based average precision (how many steps have to be made in the ranking to reach a certain relevant label, averaged by instance)
coverage
: Example and ranking based coverage (how many steps have to be made in the ranking to cover all the relevant labels, averaged by instance)
example_auc
: Example based Area Under the Curve ROC (averaged by instance)
macro_auc
: Label and ranking based Area Under the Curve ROC (macro-averaged by label)
micro_auc
: Label and ranking based Area Under the Curve ROC (micro-averaged)
one_error
: Example and ranking based one-error (how many times the top-ranked label is not a relevant label, averaged by instance)
ranking_loss
: Example and ranking based ranking-loss (how many times a non-relevant label is ranked above a relevant one, evaluated for all label pairs and averaged by instance)
Breaking ties in rankings
The additional ties_method
parameter for the ranking
function is passed to R's own rank
. It accepts the following values:
"average"
"first"
"last"
"random"
"max"
"min"
See rank
for information on the effect of each
parameter.
The default behavior in mldr corresponds to value "last"
, since this
is the behavior of the ranking method in MULAN, in order to facilitate fair
comparisons among classifiers over both platforms.
Atomical numeric vector specifying the resulting performance metric value.
Other evaluation metrics: Averaged metrics
,
Basic metrics
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | true_labels <- matrix(c(
1,1,1,
0,0,0,
1,0,0,
1,1,1,
0,0,0,
1,0,0
), ncol = 3, byrow = TRUE)
predicted_probs <- matrix(c(
.6,.5,.9,
.0,.1,.2,
.8,.3,.2,
.7,.9,.1,
.7,.3,.2,
.1,.8,.3
), ncol = 3, byrow = TRUE)
# by default, labels with same ranking are assigned ascending rankings
# in the order they are encountered
coverage(true_labels, predicted_probs)
# in the following, labels with same ranking will receive the same,
# averaged ranking
average_precision(true_labels, predicted_probs, ties_method = "average")
# the following will treat all undefined values as 0 (counting them
# for the average)
example_auc(true_labels, predicted_probs, undefined_value = 0)
# the following will ignore undefined values (not counting them for
# the average)
example_auc(true_labels, predicted_probs, undefined_value = NA, na.rm = TRUE)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.