waic.samples: Generate samples for the Widely Applicable Information...

waic.samplesR Documentation

Generate samples for the Widely Applicable Information Criterion

Description

Function to extract random samples of the Widely Applicable Information Criterion (WAIC) from a jags model.

Usage

waic.samples(model, n.iter, thin = 1, nodes="_observed_", scale=2*n, trace=FALSE, ...)

Arguments

model

A jags model object.

n.iter

Number of iterations to monitor.

thin

Thinning interval for monitors.

nodes

character vector of variable names containing nodes to be monitored. A variable name can be the name of a whole node array, e.g. "x", or subset, e.g. "x[2:5,2]". The monitored nodes will be any nodes partially or completely contained in the given variable array or subset. The default value "_observed_" monitors all observed stochastic nodes.

scale

Scaling factor for WAIC. The default scaling by twice the sample size n gives a WAIC statistic comparable with AIC and DIC.

trace

Logical flag. If TRUE then a trace of the likelihood will be returned for each monitored node in addition to the WAIC table.

...

optional arguments passed to the update method for jags model objects

Details

The waic.samples function calculates the Widely Applicable Information Criterion (WAIC) (Watanabe 2010), a predictive information criterion for model choice based on singular learning theory (Watanabe 2009).

Suppose that Y_1 \ldots Y_n are exchangeable observations that are conditionally independent given parameters \theta. The posterior predictive density q(y) is defined as:

q(y) = \text{E}_{\theta} \left[p(y \mid \theta) \mid Y \right]

The princple behind WAIC is that model fit should be measured by the Bayes generalization error BL_g, which is minus the expectation of the log predictive density for a new observation Y_{n+1}.

BL_g = \text{E} \left[ \log q(Y_{n+1}) \right]

where the expectation is over the true, unknown distribution of Y_{n+1}. In practice, this cannot be evaluated but we can approximate the Bayes generalization error by replacing the expectation with the empirical average over the observed data. This gives the Bayes training loss.

BL_t = - \frac{1}{n} \sum_{i=1}^n \log \left[ q(Y_i) \right]

The training loss is a biased estimate of the generalization error. It systematically under-estimates the generalization error because it uses the data twice: once to evaluate the posterior predictive distribution and again to evaluate the expectation over the data.

Watanabe (2009, 2010) showed that the bias of the Bayes training loss can be removed asymptotically for large sample size n by adding a penalty p_W/n to give the WAIC:

\text{WAIC} = BL_t + \frac{p_W}{n}

where

p_W = \sum_{i=1}^n \text{Var}_{\theta} \left[ \log p\left(Y_i \mid \theta \right) \mid Y \right]

The penalty p_W can be considered a measure of model complexity. In regular models with non-informative priors, p_W reduces to the dimension of the parameters \theta.

It is common practice to use version of WAIC scaled by 2n. This version of WAIC can be written:

WAIC = - 2 \sum_{i=1}^n \log \left[ q(Y_i) \right] + 2 p_W

In this form, WAIC is comparable to the Akaike Information Criterion (AIC) and the Deviance Information Criterion (Spiegelhalter et al 2002). Scaling of WAIC is controlled by the scale parameter.

Value

An object of class "JAGSwaic". This is a list containing the following elements:

waic.table

A matrix with one column for each chain and 3 rows

training_loss

The Bayes training loss multiplied by the scale parameter.

p_waic

The WAIC penalty, measuring the dimension of the model

.

waic

Penalized version of the Bayes training loss estimating the Bayes generalization error.

mcarray

A list of objects of class "mcarray" containing at least the following elements

likelihood_mean

The posterior mean of the likelihood for each node. This is the posterior predictive density evaluated at the observed value.

loglikelihood_var

The posterior variance of the likelihood for each node. This is the contribution of each node to the WAIC penalty p_waic and can also be considered a measure of local influence for each observation (Plummer 2025).

Optionally, if trace=TRUE the list includes

loglikelihood_trace

Trace of the log likelihood for each monitored node. If there are a large number of nodes this may use a lot of memory.

Author(s)

Matt Denwood, Martyn Plummer

References

Plummer, M. (2025) Bayesian measures of leverage and influence. Arxiv, https://arxiv.org/html/2503.19996

Spiegelhalter, D., N. Best, B. Carlin, and A. van der Linde (2002), Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B 64, 583-639.

Watanabe, S (2009), Algebraic Geometry and Statistical Learning Theory, Cambridge University Press.

Watanabe, S (2010) Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory, Journal of Machine Learning Research, 11, 116, 3571–3594.

See Also

dic.samples


rjags documentation built on Nov. 21, 2026, 9:10 a.m.