| waic.samples | R Documentation |
Function to extract random samples of the Widely Applicable
Information Criterion (WAIC) from a jags model.
waic.samples(model, n.iter, thin = 1, nodes="_observed_", scale=2*n, trace=FALSE, ...)
model |
A jags model object. |
n.iter |
Number of iterations to monitor. |
thin |
Thinning interval for monitors. |
nodes |
character vector of variable names containing nodes to be monitored. A variable name can be the name of a whole node array, e.g. "x", or subset, e.g. "x[2:5,2]". The monitored nodes will be any nodes partially or completely contained in the given variable array or subset. The default value "_observed_" monitors all observed stochastic nodes. |
scale |
Scaling factor for WAIC. The default scaling by twice the
sample size |
trace |
Logical flag. If TRUE then a trace of the likelihood will be returned for each monitored node in addition to the WAIC table. |
... |
optional arguments passed to the update method for jags model objects |
The waic.samples function calculates the Widely Applicable
Information Criterion (WAIC) (Watanabe 2010), a predictive information
criterion for model choice based on singular learning theory (Watanabe
2009).
Suppose that Y_1 \ldots Y_n are exchangeable
observations that are conditionally independent given parameters
\theta. The posterior predictive density q(y)
is defined as:
q(y) = \text{E}_{\theta} \left[p(y \mid \theta) \mid Y \right]
The princple behind WAIC is that model fit should be measured by
the Bayes generalization error BL_g, which is minus the
expectation of the log predictive density for a new observation
Y_{n+1}.
BL_g = \text{E} \left[ \log q(Y_{n+1}) \right]
where the expectation is over the true, unknown distribution of
Y_{n+1}. In practice, this cannot be evaluated but we can
approximate the Bayes generalization error by replacing the
expectation with the empirical average over the observed data. This
gives the Bayes training loss.
BL_t = - \frac{1}{n} \sum_{i=1}^n \log \left[ q(Y_i) \right]
The training loss is a biased estimate of the generalization error. It systematically under-estimates the generalization error because it uses the data twice: once to evaluate the posterior predictive distribution and again to evaluate the expectation over the data.
Watanabe (2009, 2010) showed that the bias of the Bayes training loss
can be removed asymptotically for large sample size n by adding a
penalty p_W/n to give the WAIC:
\text{WAIC} = BL_t + \frac{p_W}{n}
where
p_W = \sum_{i=1}^n \text{Var}_{\theta} \left[ \log p\left(Y_i \mid
\theta \right) \mid Y \right]
The penalty p_W can be considered a measure of model
complexity. In regular models with non-informative priors,
p_W reduces to the dimension of the parameters
\theta.
It is common practice to use version of WAIC scaled by
2n. This version of WAIC can be written:
WAIC = - 2 \sum_{i=1}^n \log \left[ q(Y_i) \right] + 2 p_W
In this form, WAIC is comparable to the Akaike Information Criterion
(AIC) and the Deviance Information Criterion (Spiegelhalter et al
2002). Scaling of WAIC is controlled by the scale parameter.
An object of class "JAGSwaic". This is a list containing the following elements:
waic.table |
A matrix with one column for each chain and 3 rows
.
|
mcarray |
A list of objects of class "mcarray" containing at least the following elements
Optionally, if
|
Matt Denwood, Martyn Plummer
Plummer, M. (2025) Bayesian measures of leverage and influence. Arxiv, https://arxiv.org/html/2503.19996
Spiegelhalter, D., N. Best, B. Carlin, and A. van der Linde (2002), Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B 64, 583-639.
Watanabe, S (2009), Algebraic Geometry and Statistical Learning Theory, Cambridge University Press.
Watanabe, S (2010) Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory, Journal of Machine Learning Research, 11, 116, 3571–3594.
dic.samples
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.