compute_stats_agreement: Agreement of Human and AI Decision Makers

In aihuman: Experimental Evaluation of Algorithm-Assisted Human Decision-Making

Agreement of Human and AI Decision Makers

Description

Estimate the impact of AI recommendations on the agreement between human decisions and AI recommendations using a difference-in-means estimator of an indicator 1\{D_i = A_i\}.

Usage

compute_stats_agreement(Y, D, Z, A, X = NULL)

Arguments

Y	An observed outcome (binary: numeric vector of 0 or 1).
D	An observed decision (binary: numeric vector of 0 or 1).
Z	A treatment indicator (binary: numeric vector of 0 or 1).
A	An AI recommendation (binary: numeric vector of 0 or 1).
X	Pretreatment covariate used for subgroup analysis (vector). Must be the same length as Y, D, Z, and A if provided. Default is NULL.

Value

A tibble with the following columns:

• X: Pretreatment covariate (if provided).

• agree_diff: Difference in agreement between human decisions and AI recommendations.

• agree_diff_se: Standard error of the difference in agreement.

Examples

compute_stats_agreement(
  Y = NCAdata$Y,
  D = ifelse(NCAdata$D == 0, 0, 1),
  Z = NCAdata$Z,
  A = PSAdata$DMF
)

aihuman documentation built on April 12, 2025, 1:47 a.m.