View source: R/pauc_comparison.R
| compare_pauc | R Documentation |
Statistically compares partial Area Under the Curve (pAUC) values between two lineup conditions using bootstrap-based standard errors and z-tests. Follows the methodology from pyWitness (Mickes et al., 2024).
compare_pauc(
data1,
data2,
lineup_size = 6,
max_false_id_rate = NULL,
n_bootstrap = 2000,
conf_level = 0.95,
seed = NULL,
label1 = "Condition 1",
label2 = "Condition 2",
innocent_suspect = "auto"
)
data1 |
First dataset (dataframe with target_present, identification, confidence) |
data2 |
Second dataset (same format as data1) |
lineup_size |
Integer. Number of people in lineup (default = 6) |
max_false_id_rate |
Numeric. Upper limit of the false-ID rate region over
which partial AUC is computed. Default |
n_bootstrap |
Integer. Number of bootstrap samples (default = 2000) |
conf_level |
Numeric. Confidence level for intervals (default = 0.95) |
seed |
Integer. Random seed for reproducibility (default = NULL) |
label1 |
Character. Label for first condition (default = "Condition 1") |
label2 |
Character. Label for second condition (default = "Condition 2") |
innocent_suspect |
How innocent-suspect identifications in
target-absent lineups are obtained. |
This function implements an approximate statistical test for comparing ROC curves between two conditions (e.g., simultaneous vs. sequential lineups, different retention intervals, etc.).
**Method:**
Computes pAUC for each condition (optionally up to a maximum false ID rate)
Uses bootstrap resampling to estimate standard errors
Computes Z-statistic: Z = (pAUC1 - pAUC2) / SE(pAUC1 - pAUC2)
Calculates p-value from standard normal distribution
The two datasets are treated as independent conditions. This function does not implement a paired or clustered bootstrap.
**Interpretation:**
Positive pAUC difference: Condition 1 has the larger pAUC over the shared false-ID range
Negative pAUC difference: Condition 2 has the larger pAUC over the shared false-ID range
p < 0.05: Significant difference in pAUC between conditions; otherwise there is no significant difference
The comparison covers only the shared false-ID range and uses suspect IDs only, so it is not a general statement about discriminability. A common-range pAUC can mislead when a procedure raises both culprit and innocent-suspect IDs (Smith et al., 2019).
The independent-sample bootstrap estimates uncertainty in an empirical partial-area functional; the final comparison uses a normal/Wald interval. It is approximate and can be liberal in finite samples. A data-selected cutoff, sparse high-confidence cells, or resamples that do not reach the cutoff require additional caution. It is not an exact discrimination test.
An object of class "pauc_comparison" containing:
pauc1: pAUC for condition 1
pauc2: pAUC for condition 2
pauc_diff: Difference in pAUC (pauc1 - pauc2)
se_pauc1: Bootstrap standard error for pAUC1
se_pauc2: Bootstrap standard error for pAUC2
se_diff: Standard error of the difference
z_score: Z-statistic for difference test
p_value: Two-tailed p-value
ci_diff: Confidence interval for difference
roc1: ROC object for condition 1
roc2: ROC object for condition 2
max_false_id_rate: Cutoff used for pAUC
label1, label2: Condition labels
n_bootstrap: Number of bootstrap samples used
Smith, A. M., Lampinen, J. M., Wells, G. L., Smalarz, L., & Mackovichova, S. (2019). Deviation from perfect performance measures the diagnostic utility of eyewitness lineups but partial area under the ROC curve does not. Journal of Applied Research in Memory and Cognition, 8(1), 50-59. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.jarmac.2018.09.003")}
Mickes, L., Seale-Carlisle, T. M., Chen, X., & Boogert, S. (2024). pyWitness 1.0: A python eyewitness identification analysis toolkit. Behavior Research Methods, 56, 1533-1550.
Wixted, J. T., & Mickes, L. (2012). The field of eyewitness memory should abandon probative value and embrace receiver operating characteristic analysis. Perspectives on Psychological Science, 7(3), 275-278.
# Compare two lineup procedures (halves of the example data)
data(lineup_example)
odd <- seq(1, nrow(lineup_example), by = 2)
comparison <- compare_pauc(
lineup_example[odd, ],
lineup_example[-odd, ],
label1 = "Procedure A",
label2 = "Procedure B",
n_bootstrap = 200,
seed = 123
)
print(comparison)
plot(comparison)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.