adjust_weights: Compute Pareto-smoothed Importance Weights for Alternative...

In CoryMcCartan/adjustr: Stan Model Adjustments and Sensitivity Analyses using Importance Sampling

Compute Pareto-smoothed Importance Weights for Alternative Model Specifications

Description

Given a set of new sampling statements, which can be parametrized by a data frame or list, compute Pareto-smoothed importance weights and attach them to the specification object, for further calculation and plotting.

Usage

adjust_weights(spec, object, data = NULL, keep_bad = FALSE, incl_orig = TRUE)

Arguments

spec	An object of class adjustr_spec, probably produced by make_spec, containing the new sampling sampling statements to replace their counterparts in the original Stan model, and the data, if any, by which these sampling statements are parametrized.
object	A model object, either of type stanfit, stanreg, brmsfit, or a list with two elements: model containing a CmdStanModel, and fit containing a CmdStanMCMC object.
data	The data that was used to fit the model in object. Required only if one of the new sampling specifications involves Stan data variables.
keep_bad	When FALSE (the strongly recommended default), alternate specifications which deviate too much from the original posterior, and which as a result cannot be reliably estimated using importance sampling (i.e., if the Pareto shape parameter is larger than 0.7), have their weights discarded—weights are set to NA_real_.
incl_orig	When TRUE, include a row for the original model specification, with all weights equal. Can facilitate comparison and plotting later.

Details

This function does the bulk of the sensitivity analysis work. It operates by parsing the model code from the provided Stan object, extracting the parameters and their sampling statements. It then uses R metaprogramming/tidy evaluation tools to flexibly evaluate the log density for each draw and each sampling statement, under the original and alternative specifications. From these, the function computes the overall importance weight for each draw and performs Pareto-smoothed importance sampling. All of the work is performed in R, without recompiling or refitting the Stan model.

Value

A tibble, produced by converting the provided specs to a tibble (see as.data.frame.adjustr_spec), and adding columns .weights, containing vectors of weights for each draw, and .pareto_k, containing the diagnostic Pareto shape parameters. Values greater than 0.7 indicate that importance sampling is not reliable. If incl_orig is TRUE, a row is added for the original model specification. Weights can be extracted with the pull.adjustr_weighted method. The returned object also includes the model sample draws, in the draws attribute.

References

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2015). Pareto smoothed importance sampling. arXiv preprint arXiv:1507.02646.

See Also

make_spec, summarize.adjustr_weighted, spec_plot

Examples

## Not run: 
model_data = list(
    J = 8,
    y = c(28, 8, -3, 7, -1, 1, 18, 12),
    sigma = c(15, 10, 16, 11, 9, 11, 10, 18)
)

spec = make_spec(eta ~ student_t(df, 0, 1), df=1:10)
adjust_weights(spec, eightschools_m)
adjust_weights(spec, eightschools_m, keep_bad=TRUE)

spec = make_spec(y ~ student_t(df, theta, sigma), df=1:10)
adjust_weights(spec, eightschools_m, data=model_data)
# will throw an error because `y` and `sigma` aren't provided
adjust_weights(spec, eightschools_m)

## End(Not run)

CoryMcCartan/adjustr documentation built on June 25, 2022, 4:44 p.m.