waic: Compute the WAIC for a list of scenarios
In JGCRI/gcamland: GCAM Land Allocation Module

Description Usage Arguments Details Value

The WAIC is Watanabe's "Widely Applicable Information Criterion". It uses a model's performance on the training set to estimate its out of sample performance.

1	waic(aScenarioList, weighted = TRUE)

`aScenarioList`	List of `ScenarioInfo` structures.
`weighted`	If `TRUE`, treat the scenarios as uniform samples and weight them by their posterior probability densities. Otherwise, treat them as Markov chain Monte Carlo samples and do not weight them.

To calculate the WAIC, you need a collection of samples from the parameter space. Typically these samples would be collected through a Markov chain Monte Carlo process, so that parameter combinations appear in the samples proportional to their posterior probability density. However, it is also possible to use uniform density samples, weighted by their posterior probability density. This function supports both modes.

The WAIC applies to the model as a whole. That is, it provides an estimate of the expectation deviance (what we loosely called "performance" above) when we use the moel to generate predictions for some data set not used to fit the model. These predictions are assumed to be generated by sampling from the posterior PDF of the parameters and averaging together the predictions from the sampled parameter set. If you just take the modal parameter set from the PDF, your predictions will likely be biased relative to future observations. (Note, however, that there are ways to approximate what you would have gotten by sampling without actually doing the sampling.)

The WAIC can be used to compare models with different parameterizations and even different numbers of parameters. The difference between the WAIC for different models provides an indication of the expected difference in out-of-sample deviance between the models. You can also use the WAIC values to calculate Akaike weights, which can be used for model averaging (q.v. akaike_wgt for more details)

The return value is a data frame containing WAIC results, one row for each model family represented in the input. The WAIC result table has these columns:

waic: The WAIC value for the model.
lppd: The log predictive probability density. The in-sample deviance is -2 times this value.
pwaic: The effective number of parameters in the model. This can and often will be less than the actual number of parameters (and it need not be an integer).
se: An estimate of the standard error for the WAIC value.
dwaic: Difference between a model's WAIC and the WAIC of the model with the lowest WAIC in the input set.
se.dwaic: Standard error for dwaic.
awgt: Akaike weight for each model.

The first four of these are characteristics of the individual model they are calculated for. The last three, dwaic, se.dwaic, and awgt are joint properties of the collection of models in the input.

Data frame containing a table of statistics based on WAIC calculations.

JGCRI/gcamland documentation built on Oct. 6, 2020, 5:30 p.m.