scoring_bias_test: Differential AI Scoring Bias (DASB) test.

In aiDIF: Differential Item Functioning for AI-Scored Assessments

Differential AI Scoring Bias (DASB) test.

Description

For each item, computes the change in item intercept from human to AI scoring within each group, then tests whether this scoring shift differs significantly across groups. A significant result indicates the AI scoring engine introduces a group-dependent parameter distortion — i.e., the AI does not merely re-scale all items uniformly but disfavours (or favours) one group at specific items.

Usage

scoring_bias_test(human_mle, ai_mle, fun = "d_fun3")

Arguments

human_mle	Output of simulate_aidif_data for human-scored data.
ai_mle	Output of simulate_aidif_data for AI-scored data. Must have the same item/group structure.
fun	Scaling function (passed to the internal scaling function) to use when normalising shifts. Default: "d_fun3".

Details

Estimand. Define the scoring shift in group g for item i threshold j as:

\delta_{igj} = d_{igj}^{\text{AI}} - d_{igj}^{\text{Human}}

The DASB is \delta_{i2j} - \delta_{i1j}. Under H_0: \text{DASB}_{ij} = 0 and independence across scoring conditions and groups,

\widehat{\mathrm{Var}}(\text{DASB}_{ij}) = (\sigma_{i1j}^{H})^2 + (\sigma_{i2j}^{H})^2 + (\sigma_{i1j}^{AI})^2 + (\sigma_{i2j}^{AI})^2

where each \sigma^2 is the diagonal element of the corresponding group-specific covariance matrix.

Value

A data.frame with one row per item (per threshold for polytomous items) and columns:

shift_g1

Scoring shift \delta_{i1} = d_{i1}^{AI} - d_{i1}^{H}.

shift_g2

Scoring shift \delta_{i2} = d_{i2}^{AI} - d_{i2}^{H}.

DASB

Differential AI Scoring Bias: \delta_{i2} - \delta_{i1}.

se

Standard error of DASB under the delta method.

z

Wald z-statistic.

p_val

Two-tailed p-value.

See Also

fit_aidif, ai_effect_summary

Examples

eg <- make_aidif_eg()
scoring_bias_test(eg$human, eg$ai)

aiDIF documentation built on April 22, 2026, 1:10 a.m.