pic: Predictive information criteria for Bayesian models fitted in...

In missingHE: Missing Outcome Data in Health Economic Evaluation

Predictive information criteria for Bayesian models fitted in JAGS using the funciton selection, selection_long, pattern or hurdle

Description

Efficient approximate leave-one-out cross validation (LOO), deviance information criterion (DIC) and widely applicable information criterion (WAIC) for Bayesian models, calculated on the observed data.

Usage

pic(x, criterion = "dic", module = "total")

Arguments

x	A missingHE object containing the results of a Bayesian model fitted in cost-effectiveness analysis using the function selection, selection_long, pattern or hurdle.
criterion	type of information criteria to be produced. Available choices are 'dic' for the Deviance Information Criterion, 'waic' for the Widely Applicable Information Criterion, and 'looic' for the Leave-One-Out Information Criterion.
module	The modules with respect to which the information criteria should be computed. Available choices are 'total' for the whole model, 'e' for the effectiveness variables only ('u' for longitudinal models), 'c' for the cost variables only, and 'both' for both outcome variables.

Details

The Deviance Information Criterion (DIC), Leave-One-Out Information Criterion (LOOIC) and the Widely Applicable Information Criterion (WAIC) are methods for estimating out-of-sample predictive accuracy from a Bayesian model using the log-likelihood evaluated at the posterior simulations of the parameters. If x contains the results from a longitudinal model, all parameter names indexed by "e" should be instead indexed by "u". In addition, for longitudinal models information criteria results are displayed by time and only a general approximation to the total value of the criteria and pD is given as the sum of the corresponding measures computed at each time point. DIC is computationally simple to calculate but it is known to have some problems, arising in part from it not being fully Bayesian in that it is based on a point estimate. LOOIC can be computationally expensive but can be easily approximated using importance weights that are smoothed by fitting a generalised Pareto distribution to the upper tail of the distribution of the importance weights. For more details about the methods used to compute LOOIC see the PSIS-LOO section in loo-package. WAIC is fully Bayesian and closely approximates Bayesian cross-validation. Unlike DIC, WAIC is invariant to parameterisation and also works for singular models. In finite cases, WAIC and LOO give similar estimates, but for influential observations WAIC underestimates the effect of leaving out one observation.

Value

A named list containing different predictive information criteria results and quantities according to the value of criterion. In all cases, the measures are computed on the observed data for the specific modules of the model selected in module.

d_bar

Posterior mean deviance (only if criterion is 'dic').

pD

Effective number of parameters calculated with the formula used by JAGS (only if criterion is 'dic')

.

dic

Deviance Information Criterion calculated with the formula used by JAGS (only if criterion is 'dic')

.

d_hat

Deviance evaluated at the posterior mean of the parameters and calculated with the formula used by JAGS (only if criterion is 'dic')

elpd, elpd_se

Expected log pointwise predictive density and standard error calculated on the observed data for the model nodes indicated in module (only if criterion is 'waic' or 'loo').

p, p_se

Effective number of parameters and standard error calculated on the observed data for the model nodes indicated in module (only if criterion is 'waic' or 'loo').

looic, looic_se

The leave-one-out information criterion and standard error calculated on the observed data for the model nodes indicated in module (only if criterion is 'loo').

waic, waic_se

The widely applicable information criterion and standard error calculated on the observed data for the model nodes indicated in module (only if criterion is 'waic').

pointwise

A matrix containing the pointwise contributions of each of the above measures calculated on the observed data for the model nodes indicated in module (only if criterion is 'waic' or 'loo').

pareto_k

A vector containing the estimates of the shape parameter k for the generalised Pareto fit to the importance ratios for each leave-one-out distribution calculated on the observed data for the model nodes indicated in module (only if criterion is 'loo'). See loo for details about interpreting k.

sum_dic

DIC value calculated by summing up all model dic evaluated at each time point (only for longitudinal models). Similar estimates can are obtained also for the other criteria, either sum_waic or sum_looic.

sum_pdic

DIC value calculated by summing up all model effective number of parameter estimates based on dic evaluated at each time point (only for longitudinal models). Similar estimates can are obtained also for the other criteria, either sum_pwaic or sum_plooic.

Author(s)

Andrea Gabrio

References

Plummer, M. JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. (2003).

Vehtari, A. Gelman, A. Gabry, J. (2016a) Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. Advance online publication.

Vehtari, A. Gelman, A. Gabry, J. (2016b) Pareto smoothed importance sampling. ArXiv preprint.

Gelman, A. Hwang, J. Vehtari, A. (2014) Understanding predictive information criteria for Bayesian models. Statistics and Computing 24, 997-1016.

Watanable, S. (2010). Asymptotic equivalence of Bayes cross validation and widely application information criterion in singular learning theory. Journal of Machine Learning Research 11, 3571-3594.

See Also

jags, loo, waic

Examples

 
# For examples see the function \code{\link{selection}}, \code{\link{selection_long}},
# \code{\link{pattern}} or \code{\link{hurdle}}
# 
# 

missingHE documentation built on March 31, 2023, 10:27 p.m.