IRT recovery, SBC, and misspecification evidence

In eyeprocess: Harmonize Eye-Tracking, Pupillometry, Biometrics, and Psychometric Process Data

knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(eyeprocess)

Simulation-based calibration (SBC) and parameter recovery answer different software-validation questions. Recovery checks whether fitted estimates reproduce known generating quantities under declared scenarios; SBC checks calibration of posterior computation under the declared generative model.

design <- eyeprocess_irt_recovery_design(sample_size = 250L, n_items = 12L,
                                          missing_rate = c(0,.15), testlet_sd = c(0,.35),
                                          replications = 5L, seed = 20260811L)
head(design)
eyeprocess_irt_misspecification_suite()

Exact recovery fitting currently requires mirt; absence produces a gated result. SBC rank summaries reuse eyeprocess's rank-diagnostic infrastructure.

The SBC workflow follows the logic of Talts et al., Validating Bayesian Inference Algorithms with Simulation-Based Calibration: https://arxiv.org/abs/1804.06788.

Simulation-based calibration view

The figure below is generated from a small deterministic known-item simulation. It is a computational calibration diagnostic under the declared generative model, not evidence of empirical model adequacy.

viz_items <- data.frame(
  item_id = paste0('I', 1:8),
  a = seq(0.8, 1.5, length.out = 8),
  b = seq(-1.5, 1.5, length.out = 8),
  c = 0,
  d = 1
)

viz_sbc <- eyeprocess::run_eyeprocess_irt_ability_sbc(
  items = viz_items,
  replications = 20L,
  posterior_draws = 19L,
  theta_grid = seq(-5, 5, length.out = 201L),
  seed = 902L
)

stopifnot(
  inherits(viz_sbc, 'eye_irt_sbc_evidence')
)

plot(viz_sbc)

eyeprocess documentation built on Sept. 28, 2026, 5:08 p.m.