varimp: Permutation Variable Importance for a GRM Forest

In grmtree: Recursive Partitioning for Graded Response Models

Permutation Variable Importance for a GRM Forest

Description

Quantifies how much each partitioning variable contributes to detected differential item functioning, by measuring the loss in out-of-bag marginal log-likelihood when that variable's values are randomly permuted.

Usage

varimp(
  forest,
  method = "permutation",
  nperm = 1L,
  quadpts = 61L,
  verbose = FALSE,
  seed = NULL,
  n_cores = 1L
)

Arguments

forest	A grmforest object from grmforest().
method	Importance type. Currently only "permutation".
nperm	Number of permutations averaged per variable per tree (default: 1). Values of 3–10 reduce Monte Carlo noise at proportional cost.
quadpts	Number of quadrature points used to integrate the latent trait out of the likelihood (default: 61).
verbose	Logical. Report progress (default: FALSE).
seed	Random seed for the permutations (default: NULL).
n_cores	Number of cores used to evaluate trees in parallel (default: 1). As in grmforest(), all permutation draws are generated in the master process first, so results are independent of n_cores.

Details

For each tree, the out-of-bag rows are passed down the tree and each respondent is scored under the graded response model held in the terminal node they land in. The score is the marginal log-likelihood, integrating the latent trait against a standard normal:

\ell = \sum_i \log \int \prod_j P(Y_{ij} = y_{ij} \mid \theta) \phi(\theta) \, d\theta

evaluated by quadrature. The variable is then permuted within the out-of-bag rows, respondents are re-routed through the tree, and the likelihood is recomputed. Importance is the mean across trees of the paired difference (baseline minus permuted) for that tree, which has the same expectation as differencing two separate averages but lower variance and no sensitivity to which trees happen to be usable.

Category probabilities are obtained from mirt::probtrace() rather than reconstructed from coefficients, so the correct parameterization is applied for whatever was fitted.

Value

A named numeric vector of class varimp, sorted decreasing. Values are on the log-likelihood scale: larger means the variable matters more. Values near zero mean the variable carries no information about item parameter instability; small negative values are ordinary sampling noise around zero.

Sparse response categories

In large instruments some items may have categories that are empty within a particular terminal node, in which case mirt collapses them and the node model's category count no longer matches the full dataset. Such items are skipped for that node, with a single note at the end reporting how many were affected. Because importance uses paired within-tree differences, this reduces precision but does not bias the result. Persistent notes are a signal to collapse sparse categories deliberately before fitting, or to raise minbucket.

See Also

grmtree fits a Graded Response Model Tree, grmforest for GRM Forests, grmforest.control creates a control object for grmforest, plot.varimp creates a bar plot of variable importance scores

Examples

library(grmtree)
library(hlt)
data("asti", package = "hlt")
asti$resp <- data.matrix(asti[, 1:4])

## Fit the GRM Forest
forest <- grmforest(resp ~ gender + group, data = asti,
control = grmforest.control(n_tree = 10, seed = 123))

importance <- varimp(forest, seed = 123)

## Print and plot the variable importance scores
print(importance)
plot(importance)

#' ## Variable importance for a longitudinal forest (same call)
  data("grmtree_long_data", package = "grmtree")
  items_t1 <- c("MOS_Listen", "MOS_Info", "MOS_Advice_Crisis", "MOS_Confide",
                "MOS_Advice_Want", "MOS_Fears", "MOS_Personal", "MOS_Understand")
  ld <- prepare_longitudinal_data(
    data = grmtree_long_data, items_t1 = items_t1,
    items_t2 = paste0(items_t1, "_year1"),
    covariates = c("sex", "age", "education"))
  lforest <- grmforest(resp_wide ~ sex + age + education, data = ld,
    control = grmforest.control(n_tree = 20, seed = 123,
                control = grmtree.control(minbucket = 200)),
    tree_fun = longitudinal_grmtree, tree_args = list(n_items = 8))
  importance <- varimp(lforest, seed = 123)
  print(importance)

grmtree documentation built on Sept. 2, 2026, 1:07 a.m.