report.compare.loo: Reporting Bayesian Model Comparison

In report: Automated Reporting of Results and Statistical Models

Reporting Bayesian Model Comparison

Description

Automatically report the results of Bayesian model comparison using the loo package.

Usage

## S3 method for class 'compare.loo'
report(x, include_IC = TRUE, include_ENP = FALSE, ...)

Arguments

x	An object of class brms::loo_compare.
include_IC	Whether to include the information criteria (IC).
include_ENP	Whether to include the effective number of parameters (ENP).
...	Additional arguments (not used for now).

Details

The rule of thumb is that the models are "very similar" if |elpd_diff| (the absolute value of elpd_diff) is less than 4 (Sivula, Magnusson and Vehtari, 2020). If superior to 4, then one can use the SE to obtain a standardized difference (Z-diff) and interpret it as such, assuming that the difference is normally distributed. The corresponding p-value is then calculated as 2 * pnorm(-abs(Z-diff)). However, note that if the raw ELPD difference is small (less than 4), it doesn't make much sense to rely on its standardized value: it is not very useful to conclude that a model is much better than another if both models make very similar predictions.

Value

Objects of class report_text().

Examples

library(brms)

m1 <- brms::brm(mpg ~ qsec, data = mtcars)
m2 <- brms::brm(mpg ~ qsec + drat, data = mtcars)
m3 <- brms::brm(mpg ~ qsec + drat + wt, data = mtcars)

x <- brms::loo_compare(
  brms::add_criterion(m1, "loo"),
  brms::add_criterion(m2, "loo"),
  brms::add_criterion(m3, "loo"),
  model_names = c("m1", "m2", "m3")
)
report(x)
report(x, include_IC = FALSE)
report(x, include_ENP = TRUE)

report documentation built on Sept. 11, 2024, 8:47 p.m.