mlm_sensitivity: Robustness diagnostics for cross-level interaction effects

In mlmoderator: Probing, Plotting, and Interpreting Multilevel Interaction Effects

Robustness diagnostics for cross-level interaction effects

Description

Assesses the stability of a cross-level interaction effect using two MLM-appropriate diagnostics:

Usage

mlm_sensitivity(
  model,
  pred,
  modx,
  alpha = 0.05,
  icc_range = c(0.01, 0.4),
  icc_grid = 50L,
  loco = TRUE,
  conf.level = 0.95,
  verbose = FALSE
)

Arguments

model	An lmerMod object with a two-way interaction between pred and modx.
pred	Character scalar. Focal predictor name.
modx	Character scalar. Moderator name.
alpha	Significance level. Default 0.05.
icc_range	Numeric vector of length 2. Range of ICC values to evaluate. Default c(0.01, 0.40).
icc_grid	Integer. Number of ICC values in the grid. Default 50.
loco	Logical. Run leave-one-cluster-out analysis? Default TRUE. Set to FALSE for large datasets where refitting is slow.
conf.level	Confidence level. Default 0.95.
verbose	Logical. Print progress during LOCO refitting? Default FALSE.

Details

1. ICC-shift robustness – how do the interaction SE and Johnson-Neyman boundary change if the intraclass correlation were different from what was observed? This is relevant because the ICC determines the effective sample size at level 2, which directly governs precision of cross-level interaction estimates.

2. Leave-one-cluster-out (LOCO) stability – refit the model dropping one cluster at a time and track how the interaction coefficient moves. This is nonparametric, makes no distributional assumptions, and directly answers: "Is this finding driven by a small number of influential clusters?"

Value

An object of class mlm_sensitivity with components:

• icc_shift: data frame of interaction SE, t, p, significance, and approximate JN boundary across the ICC grid.

• loco: data frame with one row per cluster giving the interaction coefficient, SE, t, p, and Cook's-distance-style influence measure when that cluster is omitted. NULL if loco = FALSE.

• robustness_index: proportion of the ICC range where the interaction remains significant.

• observed: list of observed model statistics.

• Metadata: pred, modx, alpha, icc_range, int_term.

Scope

These are robustness diagnostics, not a full causal sensitivity analysis. They do not quantify the strength of unmeasured confounding needed to explain away the interaction – that requires a level-2-aware omitted variable bound that is currently under development as a separate methodological contribution. See vignette("robustness-diagnostics") for interpretation guidance.

Examples

# Use a small dataset for fast execution
set.seed(42)
n_j <- 20; n_i <- 10
dat_small <- data.frame(
  y    = rnorm(n_j * n_i),
  x    = rnorm(n_j * n_i),
  m    = rep(rnorm(n_j), each = n_i),
  grp  = factor(rep(seq_len(n_j), each = n_i))
)
dat_small$y <- dat_small$y + 0.5 * dat_small$x * dat_small$m
mod_small <- lme4::lmer(y ~ x * m + (1 + x | grp), data = dat_small,
                        control = lme4::lmerControl(optimizer = "bobyqa"))

# ICC-shift only (fast)
sens <- mlm_sensitivity(mod_small, pred = "x", modx = "m", loco = FALSE)
print(sens)

# Full diagnostics including LOCO (20 clusters - fast)

sens_full <- mlm_sensitivity(mod_small, pred = "x", modx = "m")
plot(sens_full)

mlmoderator documentation built on April 4, 2026, 1:07 a.m.