bfFixefLMER_F.fnc: Back-fits an LMER model on p-values from ANOVA, llrt, AIC,...

In LMERConvenienceFunctions: Model Selection and Post-Hoc Analysis for (G)LMER Models

Description

This function back-fits an initial LMER model either on upper- or lower-bound p-values obtained from function pamer.fnc, log-likelihood ratio testing (LLRT), AIC, BIC, relLik.AIC, or relLik.BIC. Note that this function CANNOT be used with generalized linear mixed-effects models (glmers).

Usage

bfFixefLMER_F.fnc(model, item = FALSE, 
method = c("F", "llrt", "AIC", "BIC", "relLik.AIC", 
"relLik.BIC"), threshold = NULL, alpha = NULL, 
alphaitem = NULL, prune.ranefs = TRUE, 
p.value = "upper", set.REML.FALSE = TRUE, 
keep.single.factors=FALSE, reset.REML.TRUE = TRUE, 
log.file = NULL)

Arguments

model	A mer object (fitted by function lmer). Note that this function cannot be used with generalized linear mixed-effects models (glmers).
item	Whether or not to evaluate the addition of by-item random intercepts to the model, evaluated by way of log-likelihood ratio test. Either FALSE (the default) or the column name (quoted) of the item identifier (e.g., "Item", or "Word").
method	Backfitting method. One of "F" (p-value), "llrt", "AIC", "BIC", "relLik.AIC", or "relLik.BIC" (relative likelihood, see function relLik). Defaults to F. You can find information regarding differences between AIC and BIC from http://methodology.psu.edu/eresources/ask/sp07.
threshold	Method-specific threshold for parameter selection. It refers to alpha in the case of "F" and "llrt", to the minimum reduction in likelihood in the case of "AIC" and "BIC", or to the minimum difference in probability in the case of "relLik.AIC" and "relLik.BIC". Defaults NULL, which means 0.05 for "F" and "llrt", 5 for "AIC" and "BIC", and 4 for "relLik.AIC" and "relLik.BIC".
alpha	If the method is F, it is the p-value (from pamer.fnc) above which a model term is dropped. In this case, it defaults to the value passed to argument threshold, i.e., 0.05. Otherwise it is the p-value threshold above which a test (see method) is performed between a model with the term under consideration and a simpler model without it (in this case, defaults to 0, i.e. all terms will be tested).
alphaitem	Alpha value for the evaluation of by-item random intercepts. Defaults to 0.05 or to the specified threshold.
prune.ranefs	Logical. Whether to remove any random effect for which its variable is not also present in the fixed effects structure (with the exception of the grouping variables such as "Subjects" and "Items"). Defaults to TRUE. For example, if the random effects structure contains the terms Condition + ROI + Group, and the random effects structure contains the terms (1 | Subject) + (0 + TrialNum | Subject), the random effect (0 + TrialNum | Subject) will be pruned from the model given that it is not in the model's fixed effects structure.
p.value	If method = "F", whether to use upper-bound (“upper”; the default) or lower-bound (“lower”) p-values during backfitting.
set.REML.FALSE	Logical. Whether or not to set REML to FALSE. Defaults to TRUE.
keep.single.factors	Logical. Whether or not main effects are kept (not subjected to testing and reduction). Defaults to FALSE.
reset.REML.TRUE	Logical. Whether or not to re-set the back-fitted model to REML = TRUE.
log.file	Whether a back-fitting log should be saved. Defaults to NULL, which means that a log is saved in a temporary folder with the file name file.path(tempdir(), paste("bfFixefLMER_F_log_", gsub(":", "-", gsub(" ", "_", date())), ".txt", sep = "")). The path and file name of the log can be changed to whatever the use wishes. Set to FALSE to disable.

Details

The back-fitting process works as follows:

1. If argument method is not set to F, REML is set to FALSE;

2. First consider only highest-order interaction model terms:

   1. If method is F, the model term with the highest ANOVA p-value is identified. If this p-value is higher than alpha,the model term is removed and a new model is fitted. This is repeated for each model term that has a p-value higher than the alpha value. The algorithm then moves on to step (b). If method is not F, the model term with the lowest p-value is identified and the following is evaluated:

      1. A new model without this model term is fitted;

      2. The more complex and simpler models are compared by way of a log-likelihood ratio test in case method is "llrt", by way of AIC or BIC values in case method is "AIC" or "BIC", or by calculating the relLik based on AIC or BIC in case method is "relLik.AIC" or "relLik.BIC". If the result determines that the term under consideration does not increase model fit, it is removed; otherwise it is kept.

      3. Move on to the next model term with the smallest p-value smaller than alpha and repeat steps (i)–(iii).

   2. Once all highest-order interaction terms have been evaluated, go down to the second highest order interactions: Repeat steps (ai)–(aiii) with the following addition: If a term would be removed from the model, but it is part of a high-order interaction, keep it. Once all terms of the interaction level have been evaluated, move down to the next lower-order level until main effects have been evaluated, after which the process stops. If keep.single factors = TRUE, the process stops after the evaluation of all interaction terms.

3. If argument method is set to something else other than "F", set reset.REML.TRUE to TRUE (default) unless otherwise specified.

In brief, if method is set to "F", a term remains in the model if its p-value is equal to or greater than alpha; if method is set to something else, a term remains in the model if

1. its p-value from the ANOVA is equal to or smaller than alpha;

2. it significantly increases model fit as determined by the specified method;

3. it is part of a significant higher-order interaction term.

This backfitting method was used in Newman, Tremblay, Nichols, Neville, and Ullman (2012). If factorial terms are included in the initial model, back-fitting on F is recommended.

Value

A mer model with back-fitted fixed effects is returned and a log of the back-fitting process is printed on screen and (by default) in a log file in a temporary file.

Warnings

Upper-bound p-values can be anti-conservative, while lower-bound p-values can be conservative. See function pamer.fnc.

Note

If you get this error:

Error in model.frame.default(data = ..2, formula = log_Segment_Duration ~  :
The ... list does not contain 2 elements 

It is probably because you updated the model using function update and the data now appears as data = ..2 or something similar to this. You can check this by typing model@call. If this is the case, re-fit your model as lmer(DV ~ IV + IV + (RANEF), data = dat).

Author(s)

Antoine Tremblay, Statistics Canada, [e-mail] and Johannes Ransijn [e-mail].

References

Newman, A.J., Tremblay, A., Nichols, E.S., Neville, H.J., and Ullman, M.T. (2012). The Influence of Language Proficiency on Lexical Semantic Processing in Native and Late Learners of English. Journal of Cognitive Neuroscience, 25, 1205–1223.

See Also

bfFixefLMER_t.fnc; ffRanefLMER.fnc; fitLMER.fnc; mcposthoc.fnc; pamer.fnc; mcp.fnc; relLik; romr.fnc; perSubjectTrim.fnc.

Examples

# see example in LMERConvenienceFunctions help page.

Example output

Warning messages:
1: In rgl.init(initValue, onlyNULL) : RGL: unable to open X11 display
2: 'rgl_init' failed, running with rgl.useNULL = TRUE 
3: .onUnload failed in unloadNamespace() for 'rgl', details:
  call: fun(...)
  error: object 'rgl_quit' not found 

LMERConvenienceFunctions documentation built on Oct. 23, 2020, 5:12 p.m.