ICC_models: Intraclass correlation coefficient (ICC) of BERT models.

In FMAT: The Fill-Mask Association Test

Intraclass correlation coefficient (ICC) of BERT models.

Description

Interrater agreement of log probabilities (treated as "ratings"/rows) among BERT language models (treated as "raters"/columns), with both row and column as ("two-way") random effects.

Usage

ICC_models(data, type = "agreement", unit = "average")

Arguments

data	Raw data returned from FMAT_run() (with variable prob) or its summarized data obtained with summary.fmat() (with variable LPR).
type	Interrater "agreement" (default) or "consistency".
unit	Reliability of "average" scores (default) or "single" scores.

Value

A data.frame of ICC.

FMAT documentation built on Jan. 12, 2026, 9:08 a.m.