clbr_curve: Calibration Curve
In chuvanan/calibcurve: Calibration of Predicted Probabilities

Description Usage Arguments Details Value Multiclass Relevant Level Author(s) Examples

calibration_curve() computes the true and predicted probabilities for a calibration curve.

calibration_curve(data, ...)

## S3 method for class 'data.frame'
calibration_curve(
  data,
  truth,
  ...,
  n_bins = 10L,
  scale_estimate = FALSE,
  discretise_strategy = c("uniform", "quantile"),
  na_rm = TRUE
)

calibration_curve_vec(
  truth,
  estimate,
  n_bins = 10L,
  scale_estimate = FALSE,
  discretise_strategy = c("uniform", "quantile"),
  na_rm = TRUE,
  ...
)

## S3 method for class 'clbr_df'
autoplot(object, ...)

`data`	A `data.frame` containing the `truth` and `estimate` columns.
`...`	Not currently used
`truth`	The column identifier for the true class results (that is a `factor`). This should be an unquoted column name although this argument is passed by expression and supports quasiquotation (you can unquote column names). For `_vec()` functions, a factor vector.
`n_bins`	Number of bins to discretize the `[0,1]` interval. Default is 10.
`scale_estimate`	A `logical` value indicating whether `estimate` should be normalised into the `[0,1]` interval.
`discretise_strategy`	Strategy used to define the widths of the bins which is either 'uniform' (default) or 'quantile'. If 'uniform', the bins have idential widths. If 'quantile', the bins have the same number of samples.
`na_rm`	A `logical` value indicating whether NA values should be stripped before the computation proceeds.
`estimate`	The column identifier for the predicted results (that is also `numeric`). As with `truth` this can be specified different ways but the primary method is to use an unquoted variable name. For `_vec()` functions, a `numeric` vector.
`object`	The `clbr_df` data frame returned from `calibration_curve()`

The function takes on inputs coming from a binary classifier.

Calibration curve is also known as reliability diagram. This function is named as so to be akin to scikit-learn's calibration_curve method.

Quotes from Niculescu-Mizil & Caruana (2005) with minor modifications: First, the predicted values (probabilities) is discretized into ten bins (default, can be changed). Cases with predicted values between 0 and 0.1 fall in the first bin, between 0.1 and 0.2 in the second bin, etc. For each bin, the mean predicted value is plotted against the true fraction of positive cases.

There is a ggplot2::autoplot() method for quickly visualising the curve. This works for binary and multiclass output, and also works with grouped data (i.e. from resamples).

A tibble with clbr_df or clbr_grouped_df having columns .frac_positive and .mean_predicted

If a multiclass truth column is provided, a one-vs-all approach will be taken to calculate multiple curves, one per level. In this case, there will be an additional column, .level, identifying the "one" column in the one-vs-all calculation.

There is no common convention on which factor level should automatically be considered the "event" or "positive" result. In yardstick, the default is to use the first level. To change this, a global option called yardstick.event_first is set to TRUE when the package is loaded. This can be changed to FALSE if the last level of the factor is considered the level of interest by running: options(yardstick.event_first = FALSE). For multiclass extensions involving one-vs-all comparisons (such as macro averaging), this option is ignored and the "one" level is always the relevant result.

An Chu

## Not run: 

library(dplyr)
library(ggplot2)

data("two_class_example", package = "yardstick")

two_class_example %>%
    calibration_curve(truth, Class1)

two_class_example %>%
   calibration_curve(truth, Class1) %>%
   autoplot()


## End(Not run)