ai_effect_summary: Summarise the effect of AI scoring on DIF flagging.

In aiDIF: Differential Item Functioning for AI-Scored Assessments

Summarise the effect of AI scoring on DIF flagging.

Description

Compares the DIF flagging patterns from human and AI scoring conditions and classifies each item as: "stable_clean" (not flagged in either), "stable_dif" (flagged in both), "introduced" (flagged only under AI), "masked" (flagged only under human), or "new_direction" (flagged in both but bias reverses sign).

Usage

ai_effect_summary(dif_human, dif_ai, alpha = 0.05)

Arguments

dif_human	A data.frame returned by fit_aidif for the human scoring condition.
dif_ai	A data.frame returned by fit_aidif for the AI scoring condition.
alpha	Significance threshold for flagging. Default: 0.05.

Value

A data.frame with one row per item/threshold and columns:

human_delta

Estimated DIF effect under human scoring.

ai_delta

Estimated DIF effect under AI scoring.

human_flag

Logical: flagged under human scoring?

ai_flag

Logical: flagged under AI scoring?

status

Classification (see Description).

See Also

scoring_bias_test, fit_aidif

Examples

eg <- make_aidif_eg()
mod <- fit_aidif(eg$human, eg$ai)
ai_effect_summary(mod$dif_human, mod$dif_ai)

aiDIF documentation built on April 22, 2026, 1:10 a.m.