WAIC: calculates the WAIC

Description Usage Arguments Details Note Author(s) References See Also Examples

View source: R/WAIC.R

Description

calculates the WAIC

Usage

1
WAIC(bayesianOutput, numSamples = 1000, ...)

Arguments

bayesianOutput

an object of class BayesianOutput. Must implement a log-likelihood density function that can return point-wise log-likelihood values ("sum" argument).

numSamples

the number of samples to calculate the WAIC

...

optional values to be passed on the the getSample function

Details

The WAIC is constructed as

WAIC = -2 * (lppd - pWAIC)

The lppd (log pointwise predictive density), defined in Gelman et al., 2013, eq. 4 as

lpdd = Σi=1n log 1/S Σs=1S p(yi | Θs)

The value of pWAIC can be calculated in two ways, the method used is determined by the method argument.

Method 1 is defined as,

pWAIC1 = 2 Σi=1n {log[1/S Σs=1S p(yi | Θs)] - 1/S Σs=1S log p(yi | Θs)}

Method 2 is defined as,
pWAIC2 = 2 Σi=1n Vs=1S[log p(yi | Θs)]

where Vs=1S is the sample variance.

Note

The function requires that the likelihood passed on to BayesianSetup contains the option sum = T/F, with defaul F. If set to true, the likelihood for each data point must be returned.

Author(s)

Florian Hartig

References

Gelman, Andrew and Jessica Hwang and Aki Vehtari (2013), "Understanding Predictive Information Criteria for Bayesian Models," http://www.stat.columbia.edu/~gelman/research/unpublished/waic_understand_final.pdf.

Watanabe, S. (2010). "Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory", Journal of Machine Learning Research, http://www.jmlr.org/papers/v11/watanabe10a.html.

See Also

DIC, MAP, marginalLikelihood

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
bayesianSetup <- createBayesianSetup(likelihood = testDensityNormal, 
                                     prior = createUniformPrior(lower = rep(-10,2),
                                                                upper = rep(10,2)))

# likelihood density needs to have option sum = FALSE

testDensityNormal(c(1,1,1), sum = FALSE)
bayesianSetup$likelihood$density(c(1,1,1), sum = FALSE)
bayesianSetup$likelihood$density(matrix(rep(1,9), ncol = 3), sum = FALSE)

# running MCMC

out = runMCMC(bayesianSetup = bayesianSetup)

WAIC(out)

Example output

[1] -1.418939 -1.418939 -1.418939
[1] -1.418939 -1.418939 -1.418939
          [,1]      [,2]      [,3]
[1,] -1.418939 -1.418939 -1.418939
[2,] -1.418939 -1.418939 -1.418939
[3,] -1.418939 -1.418939 -1.418939
sh: 1: cannot create /dev/null: Permission denied
sh: 1: cannot create /dev/null: Permission denied

 Running DEzs-MCMC, chain  1 iteration 300 of 10002 . Current logp  -12.15722 -11.98177 -8.437552 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 600 of 10002 . Current logp  -8.715568 -9.066453 -7.881422 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 900 of 10002 . Current logp  -8.228403 -8.868031 -8.040263 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 1200 of 10002 . Current logp  -8.184607 -9.713978 -8.819776 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 1500 of 10002 . Current logp  -8.158647 -8.422407 -8.702908 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 1800 of 10002 . Current logp  -8.199527 -8.866103 -8.173781 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 2100 of 10002 . Current logp  -10.87249 -11.24591 -8.818 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 2400 of 10002 . Current logp  -9.024806 -8.003525 -10.13123 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 2700 of 10002 . Current logp  -8.032245 -7.94281 -8.101499 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 3000 of 10002 . Current logp  -8.314553 -8.793704 -9.312594 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 3300 of 10002 . Current logp  -8.74968 -8.957303 -8.742931 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 3600 of 10002 . Current logp  -9.085762 -7.888063 -9.775821 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 3900 of 10002 . Current logp  -8.810754 -8.194361 -8.597326 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 4200 of 10002 . Current logp  -8.362503 -8.22352 -8.119554 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 4500 of 10002 . Current logp  -7.901355 -9.87626 -8.303011 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 4800 of 10002 . Current logp  -8.523721 -8.49801 -9.397582 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 5100 of 10002 . Current logp  -9.393453 -8.15329 -8.958661 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 5400 of 10002 . Current logp  -8.134935 -9.106165 -8.083551 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 5700 of 10002 . Current logp  -8.671254 -8.010244 -8.116321 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 6000 of 10002 . Current logp  -8.435557 -9.650706 -7.862407 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 6300 of 10002 . Current logp  -8.710297 -8.562653 -8.069163 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 6600 of 10002 . Current logp  -8.120727 -8.347678 -8.789619 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 6900 of 10002 . Current logp  -8.627197 -8.406197 -8.110692 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 7200 of 10002 . Current logp  -8.336806 -9.787899 -8.626864 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 7500 of 10002 . Current logp  -9.25149 -8.542955 -8.656464 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 7800 of 10002 . Current logp  -8.31789 -8.40129 -9.491394 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 8100 of 10002 . Current logp  -7.940436 -8.315204 -8.107513 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 8400 of 10002 . Current logp  -8.028287 -8.352042 -8.005265 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 8700 of 10002 . Current logp  -8.614158 -9.457323 -7.986355 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 9000 of 10002 . Current logp  -8.092983 -8.519564 -8.415192 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 9300 of 10002 . Current logp  -8.742846 -7.937855 -8.355999 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 9600 of 10002 . Current logp  -7.988146 -9.024267 -8.668238 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 9900 of 10002 . Current logp  -8.068399 -10.77928 -8.345491 . Please wait! 

 Running DEzs-MCMC, chain  1 iteration 10002 of 10002 . Current logp  -8.063208 -8.091191 -8.351161 . Please wait! 
runMCMC terminated after 0.799seconds
$WAIC1
[1] 6.745153

$WAIC2
[1] 11.64829

$lppd
[1] -2.529287

$pWAIC1
[1] 0.8432894

$pWAIC2
[1] 3.294857

BayesianTools documentation built on Dec. 10, 2019, 1:08 a.m.