View source: R/fullroc_functions.R
| make_fullroc_data | R Documentation |
Computes full Receiver Operating Characteristic (ROC) data using ALL response categories (suspect ID, filler ID, rejection) following Smith, Yang & Wells (2020). This method distinguishes between investigator discriminability and eyewitness discriminability.
make_fullroc_data(
data,
conf_bins = NULL,
order = c("diagnosticity", "apriori"),
lineup_size = 6,
epsilon = 0.001,
innocent_suspect = "auto"
)
data |
A dataframe with the following columns:
|
conf_bins |
Numeric vector specifying confidence bin boundaries (default = NULL, which uses unique confidence values). Can specify bins like c(0, 60, 80, 100) to create low (0-60), medium (60-80), and high (80-100) confidence categories. |
order |
Character. Method for ordering response categories:
|
lineup_size |
Integer. Number of people in the lineup (default = 6) |
epsilon |
Numeric. Small value to add when FAR = 0 to avoid infinite DR (default = 0.001) |
innocent_suspect |
How innocent-suspect identifications in
target-absent lineups are obtained. |
This function implements the "full ROC" method described in Smith, Yang & Wells (2020). Unlike traditional partial ROC curves that only use suspect identifications, this method uses ALL eyewitness responses:
Suspect identifications (evidence of guilt)
Filler identifications (evidence of innocence)
Lineup rejections (evidence of innocence)
Each response type is crossed with confidence levels to create decision criteria. These are ordered by their diagnosticity ratio (HR/FAR) or by a-priori ordering, then cumulative hit and false alarm rates are computed to form the full ROC curve.
Without a designated innocent suspect, target-absent filler IDs are split into an estimated innocent-suspect share (1 / lineup size) and the remainder. This assumes the innocent suspect is chosen at the average filler rate, which holds for a fair lineup; with a biased lineup the estimated false-alarm rates are wrong, and a designated innocent suspect should be used when available.
The full ROC provides a threshold-free measure of investigator discriminability— the ability to distinguish guilty from innocent suspects using ALL available eyewitness evidence.
Note on ordering and bias: ordering cells by the sample
diagnosticity ratio (order = "diagnosticity") bows the curve upward
from noise alone, so the resulting AUC is optimistically biased — identical
target-present and target-absent response distributions yield AUC slightly
above 0.5, with the bias growing with the number of decision-by-confidence
cells and shrinking with sample size. For inference or comparisons across
conditions, prefer order = "apriori" (the theoretically fixed
ordering used by Smith, Yang & Wells, 2020, and Yang & Smith, 2023) or compare against a permutation
baseline.
A list containing:
roc_data: Dataframe with cumulative hit rates and false alarm rates
auc: Full area under the curve
order_used: The ordering method used
diagnosticity_table: Table showing non-cumulative rates and DR for each category
n_target_present: Number of target-present lineups
n_target_absent: Number of target-absent lineups
Smith, A. M., Yang, Y., & Wells, G. L. (2020). Distinguishing between investigator discriminability and eyewitness discriminability: A method for creating full receiver operating characteristic curves of lineup identification performance. Perspectives on Psychological Science, 15(3), 589-607.
data(lineup_example)
# Compute full ROC with diagnosticity ordering
fullroc_result <- make_fullroc_data(lineup_example)
print(fullroc_result$auc)
# Compute full ROC with specified confidence bins
fullroc_binned <- make_fullroc_data(lineup_example, conf_bins = c(0, 60, 80, 100))
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.