Description Usage Arguments Details Value Relevant Level Multiclass Author(s) References See Also Examples
roc_auc()
is a metric that computes the area under the ROC curve. See
roc_curve()
for the full curve.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 
data 
A 
... 
A set of unquoted column names or one or more

truth 
The column identifier for the true class results
(that is a 
options 
A 
estimator 
One of 
na_rm 
A 
event_level 
A single string. Either 
estimate 
If 
The underlying direction
option in pROC::roc()
is forced to
direction = "<"
. This computes the ROC curve assuming that the estimate
values are the probability that the "event" occurred, which is what they
are always assumed to be in yardstick.
Generally, an ROC AUC value is between 0.5
and 1
, with 1
being a
perfect prediction model. If your value is between 0
and 0.5
, then
this implies that you have meaningful information in your model, but it
is being applied incorrectly because doing the opposite of what the model
predicts would result in an AUC >0.5
.
A tibble
with columns .metric
, .estimator
,
and .estimate
and 1 row of values.
For grouped data frames, the number of rows returned will be the same as the number of groups.
For roc_auc_vec()
, a single numeric
value (or NA
).
There is no common convention on which factor level should
automatically be considered the "event" or "positive" result
when computing binary classification metrics. In yardstick
, the default
is to use the first level. To alter this, change the argument
event_level
to "second"
to consider the last level of the factor the
level of interest. For multiclass extensions involving onevsall
comparisons (such as macro averaging), this option is ignored and
the "one" level is always the relevant result.
The default multiclass method for computing roc_auc()
is to use the
method from Hand, Till, (2001). Unlike macroaveraging, this method is
insensitive to class distributions like the binary ROC AUC case.
Additionally, while other multiclass techniques will return NA
if any
levels in truth
occur zero times in the actual data, the HandTill method
will simply ignore those levels in the averaging calculation, with a warning.
Macro and macroweighted averaging are still provided, even though they are not the default. In fact, macroweighted averaging corresponds to the same definition of multiclass AUC given by Provost and Domingos (2001).
Max Kuhn
Hand, Till (2001). "A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems". Machine Learning. Vol 45, Iss 2, pp 171186.
Fawcett (2005). "An introduction to ROC analysis". Pattern Recognition Letters. 27 (2006), pp 861874.
Provost, F., Domingos, P., 2001. "Welltrained PETs: Improving probability estimation trees", CeDER Working Paper #IS0004, Stern School of Business, New York University, NY, NY 10012.
roc_curve()
for computing the full ROC curve.
Other class probability metrics:
average_precision()
,
gain_capture()
,
mn_log_loss()
,
pr_auc()
,
roc_aunp()
,
roc_aunu()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69  # 
# Two class example
# `truth` is a 2 level factor. The first level is `"Class1"`, which is the
# "event of interest" by default in yardstick. See the Relevant Level
# section above.
data(two_class_example)
# Binary metrics using class probabilities take a factor `truth` column,
# and a single class probability column containing the probabilities of
# the event of interest. Here, since `"Class1"` is the first level of
# `"truth"`, it is the event of interest and we pass in probabilities for it.
roc_auc(two_class_example, truth, Class1)
# 
# Multiclass example
# `obs` is a 4 level factor. The first level is `"VF"`, which is the
# "event of interest" by default in yardstick. See the Relevant Level
# section above.
data(hpc_cv)
# You can use the col1:colN tidyselect syntax
library(dplyr)
hpc_cv %>%
filter(Resample == "Fold01") %>%
roc_auc(obs, VF:L)
# Change the first level of `obs` from `"VF"` to `"M"` to alter the
# event of interest. The class probability columns should be supplied
# in the same order as the levels.
hpc_cv %>%
filter(Resample == "Fold01") %>%
mutate(obs = relevel(obs, "M")) %>%
roc_auc(obs, M, VF:L)
# Groups are respected
hpc_cv %>%
group_by(Resample) %>%
roc_auc(obs, VF:L)
# Weighted macro averaging
hpc_cv %>%
group_by(Resample) %>%
roc_auc(obs, VF:L, estimator = "macro_weighted")
# Vector version
# Supply a matrix of class probabilities
fold1 < hpc_cv %>%
filter(Resample == "Fold01")
roc_auc_vec(
truth = fold1$obs,
matrix(
c(fold1$VF, fold1$F, fold1$M, fold1$L),
ncol = 4
)
)
# 
# Options for `pROC::roc()`
# Pass options via a named list and not through `...`!
roc_auc(
two_class_example,
truth = truth,
Class1,
options = list(smooth = TRUE)
)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.