roc_auc | R Documentation |
roc_auc()
is a metric that computes the area under the ROC curve. See
roc_curve()
for the full curve.
roc_auc(data, ...)
## S3 method for class 'data.frame'
roc_auc(
data,
truth,
...,
estimator = NULL,
na_rm = TRUE,
event_level = yardstick_event_level(),
case_weights = NULL,
options = list()
)
roc_auc_vec(
truth,
estimate,
estimator = NULL,
na_rm = TRUE,
event_level = yardstick_event_level(),
case_weights = NULL,
options = list(),
...
)
data |
A |
... |
A set of unquoted column names or one or more
|
truth |
The column identifier for the true class results
(that is a |
estimator |
One of |
na_rm |
A |
event_level |
A single string. Either |
case_weights |
The optional column identifier for case weights.
This should be an unquoted column name that evaluates to a numeric column
in |
options |
No longer supported as of yardstick 1.0.0. If you pass something here it will be ignored with a warning. Previously, these were options passed on to |
estimate |
If |
Generally, an ROC AUC value is between 0.5
and 1
, with 1
being a
perfect prediction model. If your value is between 0
and 0.5
, then
this implies that you have meaningful information in your model, but it
is being applied incorrectly because doing the opposite of what the model
predicts would result in an AUC >0.5
.
Note that you can't combine estimator = "hand_till"
with case_weights
.
A tibble
with columns .metric
, .estimator
,
and .estimate
and 1 row of values.
For grouped data frames, the number of rows returned will be the same as the number of groups.
For roc_auc_vec()
, a single numeric
value (or NA
).
There is no common convention on which factor level should
automatically be considered the "event" or "positive" result
when computing binary classification metrics. In yardstick
, the default
is to use the first level. To alter this, change the argument
event_level
to "second"
to consider the last level of the factor the
level of interest. For multiclass extensions involving one-vs-all
comparisons (such as macro averaging), this option is ignored and
the "one" level is always the relevant result.
The default multiclass method for computing roc_auc()
is to use the
method from Hand, Till, (2001). Unlike macro-averaging, this method is
insensitive to class distributions like the binary ROC AUC case.
Additionally, while other multiclass techniques will return NA
if any
levels in truth
occur zero times in the actual data, the Hand-Till method
will simply ignore those levels in the averaging calculation, with a warning.
Macro and macro-weighted averaging are still provided, even though they are not the default. In fact, macro-weighted averaging corresponds to the same definition of multiclass AUC given by Provost and Domingos (2001).
Max Kuhn
Hand, Till (2001). "A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems". Machine Learning. Vol 45, Iss 2, pp 171-186.
Fawcett (2005). "An introduction to ROC analysis". Pattern Recognition Letters. 27 (2006), pp 861-874.
Provost, F., Domingos, P., 2001. "Well-trained PETs: Improving probability estimation trees", CeDER Working Paper #IS-00-04, Stern School of Business, New York University, NY, NY 10012.
roc_curve()
for computing the full ROC curve.
Other class probability metrics:
average_precision()
,
brier_class()
,
classification_cost()
,
gain_capture()
,
mn_log_loss()
,
pr_auc()
,
roc_aunp()
,
roc_aunu()
# ---------------------------------------------------------------------------
# Two class example
# `truth` is a 2 level factor. The first level is `"Class1"`, which is the
# "event of interest" by default in yardstick. See the Relevant Level
# section above.
data(two_class_example)
# Binary metrics using class probabilities take a factor `truth` column,
# and a single class probability column containing the probabilities of
# the event of interest. Here, since `"Class1"` is the first level of
# `"truth"`, it is the event of interest and we pass in probabilities for it.
roc_auc(two_class_example, truth, Class1)
# ---------------------------------------------------------------------------
# Multiclass example
# `obs` is a 4 level factor. The first level is `"VF"`, which is the
# "event of interest" by default in yardstick. See the Relevant Level
# section above.
data(hpc_cv)
# You can use the col1:colN tidyselect syntax
library(dplyr)
hpc_cv %>%
filter(Resample == "Fold01") %>%
roc_auc(obs, VF:L)
# Change the first level of `obs` from `"VF"` to `"M"` to alter the
# event of interest. The class probability columns should be supplied
# in the same order as the levels.
hpc_cv %>%
filter(Resample == "Fold01") %>%
mutate(obs = relevel(obs, "M")) %>%
roc_auc(obs, M, VF:L)
# Groups are respected
hpc_cv %>%
group_by(Resample) %>%
roc_auc(obs, VF:L)
# Weighted macro averaging
hpc_cv %>%
group_by(Resample) %>%
roc_auc(obs, VF:L, estimator = "macro_weighted")
# Vector version
# Supply a matrix of class probabilities
fold1 <- hpc_cv %>%
filter(Resample == "Fold01")
roc_auc_vec(
truth = fold1$obs,
matrix(
c(fold1$VF, fold1$F, fold1$M, fold1$L),
ncol = 4
)
)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.