| PPC-calibration | R Documentation |
Assess the calibration of the predictions, or predictive probabilities in relation to binary observations. See the Plot Descriptions and Details sections below and also the PPC Calibration vignette for more details and examples.
ppc_calibration_overlay(y, prep, ..., linewidth = 0.25, alpha = 0.2)
ppc_calibration_overlay_grouped(
y,
prep,
group,
...,
linewidth = 0.25,
alpha = 0.2
)
ppc_calibration(
y,
prep = NULL,
yrep = NULL,
prob = 0.95,
interval = c("confidence", "consistency"),
help_text = TRUE,
B = 200,
show_mean = TRUE,
show_qdots = TRUE,
qdots_quantiles = 100,
...,
linewidth = 1,
alpha = 0.1
)
ppc_calibration_grouped(
y,
yrep = NULL,
prep = NULL,
group,
prob = 0.95,
interval = c("confidence", "consistency"),
help_text = TRUE,
B = 200,
show_mean = TRUE,
show_qdots = TRUE,
qdots_quantiles = 100,
...,
linewidth = 1,
alpha = 0.1
)
ppc_loo_calibration(
y,
yrep,
lw = NULL,
psis_object = NULL,
prob = 0.95,
interval = c("confidence", "consistency"),
help_text = TRUE,
B = 200,
show_mean = TRUE,
show_qdots = TRUE,
qdots_quantiles = 100,
...,
linewidth = 1,
alpha = 0.1
)
ppc_loo_calibration_grouped(
y,
yrep,
lw = NULL,
psis_object = NULL,
group,
prob = 0.95,
interval = c("confidence", "consistency"),
help_text = TRUE,
B = 200,
show_mean = TRUE,
show_qdots = TRUE,
qdots_quantiles = 100,
...,
linewidth = 1,
alpha = 0.1
)
ppc_calibration_data(
y,
prep = NULL,
yrep = NULL,
group = NULL,
type = c("overlay", "interval"),
prob = 0.95,
interval = c("confidence", "consistency"),
B = 200
)
y |
A vector of observations. See Details. |
prep |
For |
... |
Currently unused. |
linewidth, alpha |
Arguments passed to geoms controlling line width and opacity. |
group |
A grouping variable of the same length as |
yrep |
An |
prob |
For |
interval |
For |
help_text |
For |
B |
For |
show_mean |
For |
show_qdots |
For |
qdots_quantiles |
For |
lw |
For |
psis_object |
For |
type |
For |
The PPC calibration functions are designed to assess the calibration of a model with binary outcomes. In this context, calibration refers to the agreement between predicted probabilities and conditional event probabilities (CEPs) see Dimitriadis et al. (2021) and Säilynoja et al. (2026) for details.
The required inputs are y, representing binary observations
(0 or 1), and either yrep or prep. Specifically,
ppc_calibration_overlay() and ppc_calibration_overlay_grouped() require
prep, while ppc_calibration(), ppc_calibration_grouped(),
ppc_loo_calibration(), and ppc_loo_calibration_grouped() accept either
prep or yrep.
prep or yrep.
A document with detailed explanations and examples is available in the vignettes.
The plotting functions return a ggplot object that can be further
customized using the ggplot2 package. The functions with suffix
_data() return the data that would have been drawn by the plotting
function.
ppc_calibration(),ppc_calibration_grouped()PAV-adjusted calibration plots showing the relationship between the
predicted event probabilities and the conditional event probabilities.
The interval parameter controls whether confidence intervals, or
consistency intervals are computed around the calibration curve.
ppc_calibration_overlay(),ppc_calibration_overlay_grouped()Overlay plots showing posterior samples of PAV-adjusted calibration curves for each posterior draw, which can be used to visually assess the uncertainty in the calibration curve.
ppc_loo_calibration(),ppc_loo_calibration_grouped()PAV-adjusted calibration plots to assess the calibration of the leave-one-out (LOO) predictive probabilities, computed by resampling each observation's posterior predictive draws using LOO importance weights.
ppc_calibration_data()Data frame containing the data underlying the calibration plots, which can
be used to build custom calibration plots. The type argument controls
whether the data frame for ppc_calibration_overlay() and its _grouped`` variant is computed (type = "overlay"), or the data frame for ppc_calibration()and its_groupedor_loo variant is computed (type = "interval"').
Dimitriadis, T., Gneiting, T., & Jordan, A. I. (2021). Stable reliability diagrams for probabilistic classifiers. Proceedings of the National Academy of Sciences, 118(8). https://doi.org/10.1073/pnas.2016191118
Säilynoja, T., Johnson, A., Martin, O., and Vehtari, A. (2026). Recommendations for visual predictive checks in Bayesian workflow. Journal of Visualization and Interaction. 1(1). doi:10.54337/jovi.v1i1.11478.
Other PPCs:
PPC-censoring,
PPC-discrete,
PPC-distributions,
PPC-errors,
PPC-intervals,
PPC-loo,
PPC-overview,
PPC-scatterplots,
PPC-test-statistics
color_scheme_set("brightblue")
# Make an example dataset of binary observations
ymin <- range(example_y_data(), example_yrep_draws())[1]
ymax <- range(example_y_data(), example_yrep_draws())[2]
y <- rbinom(length(example_y_data()), 1, (example_y_data() - ymin) / (ymax - ymin))
prep <- (example_yrep_draws() - ymin) / (ymax - ymin)
ppc_calibration_overlay(y, prep[1:50, ])
# Compare confidence vs consistency intervals
ppc_calibration(y, prep, interval = "confidence")
ppc_calibration(y, prep, interval = "consistency")
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.