tests: Specify a statistical test to apply

In simr: Power Analysis for Generalised Linear Mixed Models by Simulation

Specify a statistical test to apply

Description

Specify a statistical test to apply

Usage

fixed(
  xname,
  method = c("z", "t", "f", "chisq", "anova", "lr", "sa", "kr", "pb")
)

compare(model, method = c("lr", "pb"))

fcompare(model, method = c("lr", "kr", "pb"))

rcompare(model, method = c("lr", "pb"))

random()

Arguments

xname	an explanatory variable to test (character).
method	the type of test to apply (see Details).
model	a null model for comparison (formula).

Details

fixed:

Test a single fixed effect, specified by xname.

compare:

Compare the current model to a smaller one specified by the formula model.

fcompare, rcompare:

Similar to compare, but only the fixed/random part of the formula needs to be supplied.

random:

Test the significance of a single random effect.

Value

A function which takes a fitted model as an argument and returns a single p-value.

Methods

The method argument can be used to specify one of the following tests. Note that "z" is an asymptotic approximation for models not fitted with glmer and "kr" will only work with models fitted with lmer.

z:

Z-test for models fitted with glmer (or glm), using the p-value from summary. For models fitted with lmer, this test can be used to treat the t-values from summary as z-values, which is equivalent to assuming infinite degrees of freedom. This asymptotic approximation seems to perform well for even medium-sized data sets, as the denominator degrees of freedom are already quite large (cf. Baayen et al. 2008) even if calculating their exact value is analytically unsolved and computationally difficult (e.g. with Satterthwaite or Kenward-Roger approximations). Setting alpha=0.045 is roughly equal to the t=2 threshold suggested by Baayen et al. (2008) and helps compensate for the slightly anti-conservative approximation.

t:

T-test for models fitted with lm. Also available for mixed models when lmerTest is installed, using the p-value calculated using the Satterthwaite approximation for the denominator degrees of freedom by default. This can be changed by setting lmerTestDdf, see simrOptions.

lr:

Likelihood ratio test, using anova.

f:

Wald F-test, using car::Anova. Useful for examining categorical terms. For models fitted with lmer, this should yield equivalent results to method='kr'. Uses Type-II tests by default, this can be changed by setting carTestType, see simrOptions.

chisq:

Wald Chi-Square test, using car::Anova. Please note that while this is much faster than the F-test computed with Kenward-Roger, it is also known to be anti-conservative, especially for small samples. Uses Type-II tests by default, this can be changed by setting carTestType, see simrOptions.

anova:

ANOVA-style F-test, using anova and lmerTest::anova.lmerModLmerTest. For 'lm', this yields a Type-I (sequential) test (see anova); to use other test types, use the F-tests provided by car::Anova() (see above). For lmer, this generates Type-II tests with Satterthwaite denominator degrees of freedom by default, this can be changed by setting lmerTestDdf and lmerTestType, see simrOptions.

kr:

Kenward-Roger test, using KRmodcomp. This only applies to models fitted with lmer, and compares models with different fixed effect specifications but equivalent random effects.

pb:

Parametric bootstrap test, using PBmodcomp. This test will be very accurate, but is also very computationally expensive.

Tests using random for a single random effect call exactRLRT.

References

Baayen, R. H., Davidson, D. J., and Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.

Examples

lm1 <- lmer(y ~ x + (x|g), data=simdata)
lm0 <- lmer(y ~ x + (1|g), data=simdata)
anova(lm1, lm0)
compare(. ~ x + (1|g))(lm1)
rcompare(~ (1|g))(lm1)
## Not run: powerSim(fm1, compare(. ~ x + (1|g)))

simr documentation built on April 14, 2023, 12:24 a.m.