pp_validate function is based on the methods described in
Cook, Gelman, and Rubin (2006) for validating software developed to fit
particular Bayesian models. Here we take the perspective that models
themselves are software and thus it is useful to apply this validation
approach to individual models.
A fitted model object returned by one of the
rstanarm modeling functions. See
The number of replications to be performed.
A seed passed to Stan to use when refitting the model.
nreps times the process of simulating parameters and data
from the model and refitting the model to this simulated data. For each of
nreps replications we do the following:
Refit the model but without conditioning on the data (setting
prior_PD=TRUE), obtaining draws θ_true
from the prior distribution of the model parameters.
Given θ_true, simulate data y*
from the prior predictive distribution (calling
posterior_predict on the fitted model object obtained in step
Fit the model to the simulated outcome y*, obtaining parameters θ_post.
For any individual parameter, the quantile of the "true" parameter value with respect to its posterior distribution should be uniformly distributed. The validation procedure entails looking for deviations from uniformity by computing statistics for a test that the quantiles are uniformly distributed. The absolute values of the computed test statistics are plotted for batches of parameters (e.g., non-varying coefficients are grouped into a batch called "beta", parameters that vary by group level are in batches named for the grouping variable, etc.). See Cook, Gelman, and Rubin (2006) for more details on the validation procedure.
A ggplot object that can be further customized using the ggplot2 package.
In order to make it through
nreps replications without running
into numerical difficulties you may have to restrict the range for randomly
generating initial values for parameters when you fit the original
model. With any of rstanarm's modeling functions this can be done by
specifying the optional argument
init_r as some number less than the
default of 2.
Cook, S., Gelman, A., and Rubin, D. (2006). Validation of software for Bayesian models using posterior quantiles. Journal of Computational and Graphical Statistics. 15(3), 675–692.
color_scheme_set to change the color scheme of the
1 2 3 4 5
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.