model_performance.stanreg: Performance of Bayesian Models

In performance: Assessment of Regression Models Performance

Performance of Bayesian Models

Description

Compute indices of model performance for (general) linear models.

Usage

## S3 method for class 'stanreg'
model_performance(model, metrics = "all", verbose = TRUE, ...)

## S3 method for class 'BFBayesFactor'
model_performance(
  model,
  metrics = "all",
  verbose = TRUE,
  average = FALSE,
  prior_odds = NULL,
  ...
)

Arguments

model	Object of class stanreg or brmsfit.
metrics	Can be "all", "common" or a character vector of metrics to be computed (some of c("LOOIC", "WAIC", "R2", "R2_adj", "RMSE", "SIGMA", "LOGLOSS", "SCORE")). "common" will compute LOOIC, WAIC, R2 and RMSE.
verbose	Toggle off warnings.
...	Arguments passed to or from other methods.
average	Compute model-averaged index? See bayestestR::weighted_posteriors().
prior_odds	Optional vector of prior odds for the models compared to the first model (or the denominator, for BFBayesFactor objects). For data.frames, this will be used as the basis of weighting.

Details

Depending on model, the following indices are computed:

• ELPD: expected log predictive density. Larger ELPD values mean better fit. See looic().

• LOOIC: leave-one-out cross-validation (LOO) information criterion. Lower LOOIC values mean better fit. See looic().

• WAIC: widely applicable information criterion. Lower WAIC values mean better fit. See ?loo::waic.

• R2: r-squared value, see r2_bayes().

• R2_adjusted: LOO-adjusted r-squared, see r2_loo().

• RMSE: root mean squared error, see performance_rmse().

• SIGMA: residual standard deviation, see insight::get_sigma().

• LOGLOSS: Log-loss, see performance_logloss().

• SCORE_LOG: score of logarithmic proper scoring rule, see performance_score().

• SCORE_SPHERICAL: score of spherical proper scoring rule, see performance_score().

• PCP: percentage of correct predictions, see performance_pcp().

Value

A data frame (with one row) and one column per "index" (see metrics).

References

Gelman, A., Goodrich, B., Gabry, J., and Vehtari, A. (2018). R-squared for Bayesian regression models. The American Statistician, The American Statistician, 1-6.

See Also

r2_bayes

Examples

model <- suppressWarnings(rstanarm::stan_glm(
  mpg ~ wt + cyl,
  data = mtcars,
  chains = 1,
  iter = 500,
  refresh = 0
))
model_performance(model)

model <- suppressWarnings(rstanarm::stan_glmer(
  mpg ~ wt + cyl + (1 | gear),
  data = mtcars,
  chains = 1,
  iter = 500,
  refresh = 0
))
model_performance(model)

model <- BayesFactor::generalTestBF(carb ~ am + mpg, mtcars)

model_performance(model)
model_performance(model[3])
model_performance(model, average = TRUE)

performance documentation built on Nov. 2, 2023, 5:48 p.m.