pairwise_agreement: Pairwise reliability for a divergent panel

In grassr: Context-Conditioned Reporting for Binary Rater Reliability

Pairwise reliability for a divergent panel

Description

Computes pairwise PABAK between every pair of raters in an N \times k binary rating matrix and places each entry on the k = 2 reference surface at the pair's observed marginal. Also returns per-rater pooled-reference sensitivity and specificity — each rater's call rate against the panel-majority of the other k - 1 raters — which uses the larger panel's information about rater behavior rather than discarding it to a strictly pairwise comparison.

Usage

pairwise_agreement(ratings, axis = c("inter", "intra"))

Arguments

ratings	An N \times k binary rating matrix (rows = subjects, columns = raters), or a data.frame whose columns are 0/1 / logical / two-level factor. Same input conventions as grass_report.
axis	Character; "inter" (default) or "intra". The intra-axis path treats the columns of ratings as per-rater viewing pairs; see paper Sec.3.3 intra-rater section.

Details

This function is the recommended primary deliverable when grass_report() flags the panel as divergent; the panel-aggregate coefficients no longer summarize the panel adequately, but the pairwise matrix exposes the panel's structure directly (uniform inconsistency, sub-group clustering, or single- rater outliers).

Value

An object of class c("grass_pairwise", "list") with fields:

• pabak_matrix — k \times k symmetric numeric matrix; [i, j] is \mathrm{PABAK}_{ij}, the diagonal is 1.

• percentile_matrix — k \times k symmetric numeric (0–100); [i, j] is the surface percentile of \mathrm{PABAK}_{ij} on the k = 2 reference at the pair's observed marginal. Diagonal is NA.

• marginal_matrix — k \times k symmetric numeric; [i, j] is \hat\pi_{+, ij}. Diagonal is NA.

• band_lo_matrix — k \times k numeric matrix; lower endpoint of each pair's 95\ (v0.7.1 sweep convention). Diagonal is NA.

• band_hi_matrix — k \times k numeric matrix; upper endpoint of each pair's 95\ Diagonal is NA.

• pooled_per_rater — data frame with k rows, one per rater, columns rater, se_tilde, sp_tilde, n_pool_pos, n_pool_neg, n_pool_excluded (subjects with tied panel majority).

• sample — list with k, N, pi_hat, tau2_hat, axis.

• notes — character vector with caveats, if any (e.g., undefined pooled-reference at k = 2).

• call — the matched call.

Examples

set.seed(6)
Se <- c(0.95, 0.75, 0.95, 0.75, 0.95)
Sp <- c(0.75, 0.95, 0.75, 0.95, 0.75)
truth <- rbinom(200, 1, 0.5)
Y <- sapply(seq_along(Se),
            function(j) ifelse(truth == 1,
                               rbinom(200, 1, Se[j]),
                               rbinom(200, 1, 1 - Sp[j])))
pw <- pairwise_agreement(Y)
pw

grassr documentation built on Sept. 22, 2026, 5:08 p.m.