ripd: Residual-based Item Parameter Drift (RIPD) Detection...

In irtQ: Unidimensional Item Response Theory Modeling

Residual-based Item Parameter Drift (RIPD) Detection Framework

Description

This function computes three RIPD statistics–RIPD_{R}, RIPD_{S}, and RIPD_{RS}–for each item. RIPD_{R} captures differences in mean raw residuals between groups, which is typically indicative of uniform item parameter drift (IPD). RIPD_{S} captures differences in mean squared residuals between groups, reflecting nonuniform IPD. RIPD_{RS}, a combined chi-square-based index, is sensitive to both uniform and nonuniform IPD.

Usage

ripd(x, ...)

## Default S3 method:
ripd(
  x,
  data,
  score = NULL,
  group,
  focal.name,
  item.skip = NULL,
  D = 1,
  alpha = 0.05,
  missing = NA,
  purify = FALSE,
  purify.by = c("ripdrs", "ripdr", "ripds"),
  max.iter = 10,
  min.resp = NULL,
  method = "ML",
  range = c(-5, 5),
  norm.prior = c(0, 1),
  nquad = 41,
  weights = NULL,
  ncore = 1,
  verbose = TRUE,
  ...
)

## S3 method for class 'est_irt'
ripd(
  x,
  score = NULL,
  group,
  focal.name,
  item.skip = NULL,
  alpha = 0.05,
  missing = NA,
  purify = FALSE,
  purify.by = c("ripdrs", "ripdr", "ripds"),
  max.iter = 10,
  min.resp = NULL,
  method = "ML",
  range = c(-5, 5),
  norm.prior = c(0, 1),
  nquad = 41,
  weights = NULL,
  ncore = 1,
  verbose = TRUE,
  ...
)

## S3 method for class 'est_item'
ripd(
  x,
  group,
  focal.name,
  item.skip = NULL,
  alpha = 0.05,
  missing = NA,
  purify = FALSE,
  purify.by = c("ripdrs", "ripdr", "ripds"),
  max.iter = 10,
  min.resp = NULL,
  method = "ML",
  range = c(-5, 5),
  norm.prior = c(0, 1),
  nquad = 41,
  weights = NULL,
  ncore = 1,
  verbose = TRUE,
  ...
)

Arguments

x	A data frame containing item metadata (e.g., item parameters, number of categories, IRT model types, etc.); or an object of class est_irt obtained from est_irt(), or est_item from est_item(). See est_irt() or simdat() for more details about the item metadata. This data frame can be easily created using the shape_df() function.
...	Additional arguments passed to the est_score() function.
data	A matrix of examinees' item responses corresponding to the items specified in the x argument. Rows represent examinees and columns represent items.
score	A numeric vector containing examinees' ability estimates (theta values). If not provided, ripd() will estimate ability parameters internally before computing the RIPD statistics. See est_score() for more information on scoring methods. Default is NULL.
group	A numeric or character vector indicating group membership of examinees. The length of the vector should be the same as the number of rows in the response data matrix.
focal.name	A single numeric or character value specifying the focal group. For instance, given group = c(0, 1, 0, 1, 1) and '1' indicating the focal group, set focal.name = 1.
item.skip	A numeric vector of item indices to exclude from IPD analysis. If NULL, all items are included. Useful for omitting specific items based on prior insights.
D	A scaling constant used in IRT models to make the logistic function closely approximate the normal ogive function. A value of 1.7 is commonly used for this purpose. Default is 1.
alpha	A numeric value specifying the significance level (\alpha) for hypothesis testing using the RIPD statistics. Default is 0.05.
missing	A value indicating missing values in the response data set. Default is NA.
purify	Logical. Indicates whether to apply a purification procedure. Default is FALSE.
purify.by	A character string specifying which RIPD statistic is used to perform the purification. Available options are "ripdrs" for RIPD_{RS}, "ripdr" for RIPD_{R}, and "ripds" for RIPD_{S}.
max.iter	A positive integer specifying the maximum number of iterations allowed for the purification process. Default is 10.
min.resp	A positive integer specifying the minimum number of valid item responses required from an examinee in order to compute an ability estimate. Default is NULL.
method	A character string indicating the scoring method to use. Available options are: "ML": Maximum likelihood estimation "WL": Weighted likelihood estimation (Warm, 1989) "MAP": Maximum a posteriori estimation (Hambleton et al., 1991) "EAP": Expected a posteriori estimation (Bock & Mislevy, 1982) Default is "ML".
range	A numeric vector of length two specifying the lower and upper bounds of the ability scale. This is used for the following scoring methods: "ML", "WL", and "MAP". Default is c(-5, 5).
norm.prior	A numeric vector of length two specifying the mean and standard deviation of the normal prior distribution. These values are used to generate the Gaussian quadrature points and weights. Ignored if method is "ML" or "WL". Default is c(0, 1).
nquad	An integer indicating the number of Gaussian quadrature points to be generated from the normal prior distribution. Used only when method is "EAP". Ignored for "ML", "WL", and "MAP". Default is 41.
weights	A two-column matrix or data frame containing the quadrature points (in the first column) and their corresponding weights (in the second column) for the latent variable prior distribution. The weights and points can be conveniently generated using the function gen.weight(). If NULL and method = "EAP", default quadrature values are generated based on the norm.prior and nquad arguments. Ignored if method is "ML", "WL", or "MAP".
ncore	An integer specifying the number of logical CPU cores to use for parallel processing. Default is 1. See est_score() for details.
verbose	Logical. If TRUE, progress messages from the purification procedure will be displayed; if FALSE, the messages will be suppressed. Default is TRUE.

Details

Theoretical Background: From RDIF to RIPD

The RIPD framework directly adapts the residual-based differential item functioning (RDIF) detection framework (Lim et al., 2022; Lim & Choe, 2023) to detect item parameter drift (IPD) in computerized adaptive testing (CAT). Each RIPD statistic (RIPD_R, RIPD_S, RIPD_{RS}) mirrors its RDIF counterpart (RDIF_R, RDIF_S, RDIF_{RS}) in both computation and asymptotic theory, with one key adaptation: whereas the original RDIF framework estimates item parameters once from the pooled data of both groups, the RIPD framework uses the original, pre-calibrated item parameters from the CAT pool directly, eliminating the need for item recalibration. This makes RIPD practically scalable for operational CAT programs where recalibration is often infeasible due to sparse and non-ignorably missing response data.

IPD in CAT can be viewed as a special case of DIF across time (Lord, 1980; Veerkamp & Glas, 2000): the focal group represents current test-takers whose responses may reflect drift, while the reference group represents a baseline under invariant item parameters.

Residual Definition

For examinee i and item j, the raw residual is defined as

e_{ij} = u_{ij} - P_j(\hat{\theta}_i),

where u_{ij} is the observed binary response (0 or 1) and P_j(\hat{\theta}_i) is the model-predicted probability of a correct response under the original item parameters evaluated at the examinee's ability estimate \hat{\theta}_i. Because item responses are independent but not identically distributed (due to varying predicted probabilities across examinees with different ability levels), the asymptotic distributions of the RIPD statistics are established via Lyapunov's central limit theorem.

Three RIPD Statistics and Their Asymptotic Distributions

RIPD_R (uniform drift): Measures the difference in mean raw residuals between the focal (F) and reference (R) groups,

RIPD_{R,j} = \frac{\sum_{i \in F} e_{ij}}{N_F} - \frac{\sum_{i \in R} e_{ij}}{N_R}.

RIPD_{R,j} is most effective when the item response functions (IRFs) of the two groups differ primarily in location (e.g., drift in the difficulty b or guessing c parameter). Under the null hypothesis H_0 of no IPD, RIPD_{R,j} asymptotically follows a normal distribution with mean zero and analytically derived variance \sigma^2_{RIPD_R} = \frac{\sum_{i \in F} P_{ij}(1 - P_{ij})}{N_F^2} + \frac{\sum_{i \in R} P_{ij}(1 - P_{ij})}{N_R^2}. A standard Z-test is then applied as Z_R = RIPD_{R,j} / \sigma_{RIPD_R}.

RIPD_S (nonuniform drift): Measures the difference in mean squared residuals between groups,

RIPD_{S,j} = \frac{\sum_{i \in F} e_{ij}^2}{N_F} - \frac{\sum_{i \in R} e_{ij}^2}{N_R}.

RIPD_{S,j} captures drift that varies across the ability continuum, such as a change in the discrimination parameter a, which RIPD_{R,j} alone may not detect. Importantly, under H_0, the null mean of RIPD_{S,j} is not zero in general; it equals

\mu_{RIPD_S} = \frac{\sum_{i \in F} P_{ij}(1 - P_{ij})}{N_F} - \frac{\sum_{i \in R} P_{ij}(1 - P_{ij})}{N_R},

which is a function of the predicted probabilities in both groups and vanishes only when their ability distributions are perfectly matched (as in the RIPD setup where focal \hat{\theta} values are reused for the synthetic reference group). The asymptotic variance is also analytically derived. A standard Z-test is applied as Z_S = (RIPD_{S,j} - \mu_{RIPD_S}) / \sigma_{RIPD_S}.

RIPD_{RS} (combined): A Wald-type statistic that jointly tests for uniform and nonuniform drift within a single framework,

RIPD_{RS,j} = (\boldsymbol{\nu}_j - \boldsymbol{\mu}_j)^\top \hat{\boldsymbol{\Sigma}}_j^{-1} (\boldsymbol{\nu}_j - \boldsymbol{\mu}_j),

where \boldsymbol{\nu}_j = (RIPD_{R,j},\ RIPD_{S,j})^\top, \boldsymbol{\mu}_j = (\mu_{RIPD_R},\ \mu_{RIPD_S})^\top is the vector of null means, and \hat{\boldsymbol{\Sigma}}_j is the analytically derived 2 \times 2 covariance matrix of (RIPD_{R,j}, RIPD_{S,j}). Under H_0, RIPD_{RS,j} asymptotically follows a \chi^2 distribution with 2 degrees of freedom. RIPD_{RS} is the recommended statistic in practice because it is sensitive to both uniform and nonuniform drift and addresses potential inflation of the family-wise Type I error that arises from applying RIPD_R and RIPD_S separately to the same item.

Diagnosing the Nature of Drift

The pattern of flagging across the three statistics can help diagnose the type of drift. If an item is flagged by RIPD_R (and possibly RIPD_{RS}) but not RIPD_S, uniform drift is indicated (e.g., difficulty shift). If flagged by RIPD_S (and possibly RIPD_{RS}) but not RIPD_R, nonuniform drift is suggested (e.g., discrimination change). Items flagged by all statistics or by RIPD_{RS} alone likely exhibit mixed drift.

CAT-Specific Workflow

The RIPD procedure for CAT consists of three steps (Lim & Han, in press):

1. Focal group CAT: The current cohort of examinees takes the CAT using the operational (potentially drifted) item pool. Each examinee receives only a subset of items; the resulting response matrix is sparse. The ML ability estimates \hat{\theta} from this step are retained.

2. Synthetic reference group construction: The focal group's \hat{\theta} values are treated as true abilities. Item responses are generated from the original (pre-calibrated, drift-free) item parameters, and an independent CAT is administered. This creates a synthetic reference group that matches the focal group's ability distribution, so that any systematic differences in residuals can be attributed to IPD rather than ability confounds. The reference group size is controlled by a replication factor: e.g., 1F reuses the focal \hat{\theta} values once (N_ref = N_foc), while kF replicates them k times. Larger reference groups (3F-8F) reduce sampling variability in reference residuals and improve detection power, with marginal gains typically diminishing beyond 5F.

3. RIPD computation with purification: Residuals are computed for both groups using the original item parameters, and the three RIPD statistics are evaluated item-by-item. Items whose Z-test (for RIPD_R or RIPD_S) or chi-square test (for RIPD_{RS}) exceeds the critical value at level alpha are flagged as drifting.

Purification Procedure

When purify = TRUE, an iterative purification procedure adapted from Lim et al. (2022) is applied to mitigate the bias in ability estimates caused by drifted items–analogous to the contaminating effect of DIF items on matching variables. At each iteration:

1. The item with the most statistically significant drift statistic (smallest p-value) among currently unflagged items is identified and flagged as a potential IPD item.

2. Ability estimates are recomputed excluding all currently flagged items.

3. RIPD statistics are recalculated using the updated ability estimates for the remaining items.

The process continues until no additional items are flagged (convergence) or the maximum number of iterations (max.iter) is reached. The statistic used to drive purification is specified by purify.by; "ripdrs" (RIPD_{RS}) is recommended as it is sensitive to both types of drift.

Key Items and item.skip

In CAT contexts, many items in the pool receive few responses and thus lack sufficient data for reliable IPD analysis. The item.skip argument allows users to exclude such items (e.g., low-exposure non-key items) from the analysis. In practice, key items for RIPD evaluation are typically identified through a preliminary CAT simulation as those with the highest average exposure frequencies under the intended operational settings.

Value

This function returns a list containing four main components:

no_purify	A list of sub-objects containing the results of IPD analysis without applying a purification procedure. The sub-objects include: ipd_stat A data frame summarizing the RIPD analysis results for all items. The columns include: item ID, RIPD_{R} statistic, standardized RIPD_{R}, RIPD_{S} statistic, standardized RIPD_{S}, RIPD_{RS} statistic, p-values for RIPD_{R}, RIPD_{S}, and RIPD_{RS}, sample sizes for the reference and focal groups, and total sample size. Note that RIPD_{RS} does not have a standardized value because it is a \chi^{2}-based statistic. moments A data frame reporting the first and second moments of the RIPD statistics. The columns include: item ID, mean and standard deviation of RIPD_{R}, mean and standard deviation of RIPD_{S}, and the covariance between RIPD_{R} and RIPD_{S}. ipd_item A list of three numeric vectors identifying items flagged as drifting by each RIPD statistic: RIPD_{R}, RIPD_{S}, and RIPD_{RS}. score A numeric vector of ability estimates used to compute the RIPD statistics.
purify	A logical value indicating whether the purification procedure was applied.
with_purify	A list of sub-objects containing the results of IPD analysis with a purification procedure. The sub-objects include: purify.by A character string indicating the RIPD statistic used for purification. Possible values are "ripdr", "ripds", and "ripdrs", corresponding to RIPD_{R}, RIPD_{S}, and RIPD_{RS}, respectively. ipd_stat A data frame reporting the RIPD analysis results for all items from the final iteration. Same structure as in no_purify, with one additional column indicating the iteration number in which each result was obtained. moments A data frame reporting the moments of RIPD statistics from the final iteration. Includes the same columns as in no_purify, with an additional column for the iteration number. ipd_item A numeric vector of item indices (row positions in x) identified as IPD items across all purification iterations, sorted in ascending order. n.iter An integer indicating the total number of iterations performed during the purification process. score A numeric vector of purified ability estimates used to compute the final RIPD statistics. complete A logical value indicating whether the purification process converged. If FALSE, the maximum number of iterations was reached without convergence.
alpha	A numeric value indicating the significance level (\alpha) used in hypothesis testing for RIPD statistics.

Methods (by class)

• ripd(default): Default method for computing the three RIPD statistics using a data frame x that contains item metadata

• ripd(est_irt): An object created by the function est_irt().

• ripd(est_item): An object created by the function est_item().

Author(s)

Hwanggyu Lim [e-mail]

References

Lim, H., & Choe, E. M. (2023). Detecting differential item functioning in CAT using IRT residual DIF approach. Journal of Educational Measurement, 60(4), 626-650. \Sexpr[results=rd]{tools:::Rd_expr_doi("doi:10.1111/jedm.12366")}.

Lim, H., Choe, E. M., & Han, K. T. (2022). A residual-based differential item functioning detection framework in item response theory. Journal of Educational Measurement, 59(1), 80-104. \Sexpr[results=rd]{tools:::Rd_expr_doi("doi:10.1111/jedm.12313")}.

Lim, H., & Han, K. T. (in press). IRT residual-based approach to detecting item parameter drift in CAT. Journal of Educational and Behavioral Statistics.

See Also

rdif(), est_irt(), est_item(), simdat(), shape_df(), est_score()

Examples

## --- RIPD Example: Detecting IPD in CAT ---------------------------------
##
## Background (Lim & Han, in press):
##   In CAT-based IPD detection using RIPD, the reference group is
##   "synthetic" -- created by re-administering a CAT to examinees whose
##   true abilities are set equal to the focal group's ML theta estimates,
##   using the ORIGINAL (pre-drift) item parameters. This eliminates the
##   need for recalibration and makes RIPD directly applicable to
##   operational CAT settings.
##
## The simIPD dataset contains:
##   - foc_resp / foc_score : focal group CAT responses + ML theta estimates
##                            (IPD items: a and b each drifted by -0.5)
##   - ref_resp / ref_score : synthetic reference group CAT responses + ML
##                            theta estimates (original parameters, 1F size)
##   - item_par  : original (non-drifted) 360-item pool in irtQ format
##   - key_item  : indices of 90 key items (highly exposed in CAT)
##   - item.skip : indices of 270 non-key items (excluded from RIPD)
##   - ipd_item  : indices of 18 truly drifted items (ground truth)
## -------------------------------------------------------------------------

data(simIPD)

## Step 1. Combine focal and synthetic reference group data
##         (reference group first, then focal group -- as in the paper)
data  <- rbind(simIPD$ref_resp,  simIPD$foc_resp)
score <- c(simIPD$ref_score,     simIPD$foc_score)
group <- c(rep(0, nrow(simIPD$ref_resp)),   # 0 = reference
           rep(1, nrow(simIPD$foc_resp)))   # 1 = focal

## Step 2. Run RIPD with purification (recommended statistic: RIPD_RS)
##         item.skip excludes the 270 non-key items from analysis
ripd_result <- ripd(
  x          = simIPD$item_par,
  data       = data,
  score      = score,
  group      = group,
  focal.name = 1,           # focal group is coded as 1
  item.skip  = simIPD$item.skip,
  D          = 1.7,
  alpha      = 0.05,
  purify     = TRUE,
  purify.by  = "ripdrs",    # purify using RIPD_RS (combined statistic)
  max.iter   = 30,
  method     = "ML",
  range      = c(-5, 5)
)

## Step 3. Review RIPD_RS results (with purification)
print(ripd_result, what = "with_purify")

## Step 4. Compare detected items to ground truth
detected <- ripd_result$with_purify$ipd_item
cat("Truly drifted items (ground truth):", simIPD$ipd_item, "\n")
cat("RIPD-detected items:               ", detected, "\n")
cat("True positives:", sum(detected %in% simIPD$ipd_item), "of",
    length(simIPD$ipd_item), "\n")

## -- Note on reference group size -----------------------------------------
## This example uses a 1F reference group (n_ref = n_foc = 3,000).
## In practice, a larger reference group (3F-8F) substantially improves
## detection power. To create a 3F reference group, replicate the focal
## theta estimates and re-run the CAT simulation with original parameters:
##
##   theta_3F  <- rep(simIPD$foc_score, times = 3)   # 9,000 examinees
##   resp_3F   <- simdat(x = simIPD$item_par, theta = theta_3F, D = 1.7)
##   # ... then run CAT and call ripd() with the larger reference group
## -------------------------------------------------------------------------

irtQ documentation built on July 27, 2026, 9:08 a.m.