Function to compute Bayes factors for ANOVA designs
This function computes Bayes factors for all main-effects and interaction contrasts in an ANOVA design.
1 2 3 4 5
a formula containing all factors to include in the analysis (see Examples)
a data frame containing data for all factors in the formula
a character vector specifying which factors are random
which set of models to compare; see Details
How many Monte Carlo simulations to generate, if relevant
prior scale for standardized, reduced fixed effects. A number of preset values can be given as strings; see Details.
prior scale for standardized random effects
approximation method, if needed. See
Models, priors, and methods of computation are provided in Rouder et al. (2012).
The ANOVA model for a vector of observations y is
y = μ + X_1 θ_1 + … + X_pθ_p +ε,
where θ_1,…,θ_p are vectors of main-effect and interaction effects, X_1,…,X_p are corresponding design matrices, and ε is a vector of zero-centered noise terms with variance σ^2. Zellner and Siow (1980) inspired g-priors are placed on effects, but with a separate g-prior parameter for each covariate:
θ_1~N(0,g_1σ^2), …, θ_p~N(0,g_p σ^2).
A Jeffries prior is placed on μ and
σ^2. Independent scaled inverse-chi-square
priors with one degree of freedom are placed on
g_1,…,g_p. The square-root of the scale for
g's corresponding to fixed and random effects is given by
When a factor is treated as random, there are as many main effect terms in the vector θ as levels. When a factor is treated as fixed, the sums-to-zero linear constraint is enforced by centering the corresponding design matrix, and there is one fewer main effect terms as levels. The Cornfield-Tukey model of interactions is assumed. Details are provided in Rouder et al. (2012)
Bayes factors are computed by integrating the likelihood
with respect to the priors on parameters. The
integration of all parameters except g_1,…,g_p
may be expressed in closed-form; the integration of
g_1,…,g_p is performed through Monte Carlo
iterations is the number of
iterations used to estimate the Bayes factor.
anovaBF computes Bayes factors for either all
submodels or select submodels missing a single main
effect or covariate, depending on the argument
whichModels. If no random factors are specified,
the null model assumed by
anovaBF is the
grand-mean only model. If random factors are specified,
the null model is the model with an additive model on all
random factors, plus a grand mean. Thus,
does not currently test random factors. Testing random
factors is possible with
whichModels controls which models are
tested. Possible values are 'all', 'withmain', 'top', and
whichModels to 'all' will test
all models that can be created by including or not
including a main effect or interaction. 'top' will test
all models that can be created by removing or leaving in
a main effect or interaction term from the full model.
'bottom' creates models by adding single factors or
interactions to the null model. 'withmain' will test all
models, with the constraint that if an interaction is
included, the corresponding main effects are also
arguments, several named values are recognized: "medium",
"wide", and "ultrawide", corresponding to r scale
values of 1/2, sqrt(2)/2, and 1,
respectively. In addition,
rscaleRandom can be set
to the "nuisance", which sets r=1 (and is thus
equivalent to "ultrawide"). The "nuisance" setting is for
medium-to-large-sized effects assumed to be in the data
but typically not of interest, such as variance due to
An object of class
BFBayesFactor, containing the
computed model comparisons
anovaBF will compute Bayes factors
for all possible combinations of fixed factors and
interactions, against the null hypothesis that all
effects are 0. The total number of tests computed will be
2^(2^K - 1) for K fixed factors.
This number increases very quickly with the number of
factors. For instance, for a five-way ANOVA, the total
number of tests exceeds two billion. Even though each
test takes a fraction of a second, the time taken for all
tests could exceed your lifetime. An option is included
to prevent this:
defaults to 50,000, is the maximum number of models that
'anovaBF' will analyze at once. This can be increased by
increasing the option value.
It is possible to reduce the number of models tested by
only testing the most complex model and every restriction
that can be formed by removing one factor or interaction
whichModels argument. Setting this
argument to 'top' reduces the number of tests to
2^K-1, which is more manageable. The Bayes factor
for each restriction against the most complex model can
be interpreted as a test of the removed
'withmain' will not reduce the number of tests as much as
'top' but the results may be more interpretable, since an
interaction is only allowed when all interacting effects
(main or interaction) are also included in the model.
Richard D. Morey (email@example.com)
Gelman, A. (2005) Analysis of Variance—why it is more important than ever. Annals of Statistics, 33, pp. 1-53.
Rouder, J. N., Morey, R. D., Speckman, P. L., Province, J. M., (2012) Default Bayes Factors for ANOVA Designs. Journal of Mathematical Psychology. 56. p. 356-374.
Zellner, A. and Siow, A., (1980) Posterior Odds Ratios for Selected Regression Hypotheses. In Bayesian Statistics: Proceedings of the First Interanational Meeting held in Valencia (Spain). Bernardo, J. M., Lindley, D. V., and Smith A. F. M. (eds), pp. 585-603. University of Valencia.
lmBF, for testing specific models, and
regressionBF for the function similar to
anovaBF for linear regression models.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
## Classical example, taken from t.test() example ## Student's sleep data data(sleep) plot(extra ~ group, data = sleep) ## traditional ANOVA gives a p value of 0.00283 summary(aov(extra ~ group + Error(ID/group), data = sleep)) ## Gives a Bayes factor of about 11.6 ## in favor of the alternative hypothesis anovaBF(extra ~ group + ID, data = sleep, whichRandom = "ID", progress=FALSE) ## Demonstrate top-down testing data(puzzles) result = anovaBF(RT ~ shape*color + ID, data = puzzles, whichRandom = "ID", whichModels = 'top', progress=FALSE) result ## In orthogonal designs, the top down Bayes factor can be ## interpreted as a test of the omitted effect