View source: R/cac_functions.R
| make_cacdata | R Documentation |
Computes confidence-accuracy data for lineup identifications following Mickes (2015). CAC analysis shows the relationship between confidence and accuracy for suspect identifications.
make_cacdata(
data,
lineup_size = 6,
confidence_bins = NULL,
innocent_suspect = "auto",
base_rate = NULL
)
data |
A dataframe with the following columns:
|
lineup_size |
Integer. Number of people in lineup (default = 6) |
confidence_bins |
Numeric vector of bin edges for grouping confidence (e.g., c(0, 60, 80, 100) creates bins 0-60, 61-80, 81-100). If NULL, uses individual confidence levels. |
innocent_suspect |
How innocent-suspect identifications in
target-absent lineups are obtained. |
base_rate |
Prior probability that the suspect is guilty, used to
compute accuracy from rates rather than counts. |
CAC analysis computes, for each confidence level:
Accuracy = \frac{Correct Suspect IDs}{Correct Suspect IDs + Incorrect Suspect IDs}
Explicit target-absent suspect IDs are used when present. Otherwise, target-absent filler IDs are divided by lineup size to estimate incorrect suspect IDs. The two estimators are never added together. Dividing by lineup size assumes the innocent suspect is chosen at the average filler rate, which holds for a fair lineup; with a biased lineup the estimated false-ID rate and the accuracy are wrong, and a designated innocent suspect should be used when one is available.
The count ratio equals the probability of guilt given a suspect ID only
when the target-present and target-absent samples are the same size, which
Mickes (2015) assumes; she notes that unequal base rates make the
calculation more complicated. With unequal samples the count-based
accuracy shifts with the TP:TA split even when hit and false-ID rates are
unchanged, and the function says so in a message. Setting base_rate
computes accuracy from the hit rate HR(c) = correct IDs / n(TP) and the
false-ID rate FAR(c) = incorrect IDs / n(TA) at the stated prior, so that
base_rate = 0.5 gives the balanced-design value whatever the sample
sizes. Its standard error is then a delta-method approximation from the two
independent binomial rates; the count-based standard error is binomial over
the bin's suspect IDs.
According to Mickes (2015), CAC analysis is most relevant for triers of fact (judges/jurors) evaluating estimator variables (e.g., exposure duration, retention interval). It shows how trustworthy an ID is at different confidence levels.
A list containing:
cac_data: Dataframe with confidence levels and accuracy
overall_accuracy: Overall accuracy across all suspect IDs
n_total_suspect_ids: Correct plus incorrect suspect IDs
base_rate: The base rate used, or NULL for count-based accuracy
n_target_present, n_target_absent: Sample sizes of the two arms
Mickes, L. (2015). Receiver operating characteristic analysis and confidence-accuracy characteristic analysis in investigations of system variables and estimator variables that affect eyewitness memory. Journal of Applied Research in Memory and Cognition, 4(2), 93-102.
Juslin, P., Olsson, N., & Winman, A. (1996). Calibration and diagnosticity of confidence in eyewitness identification: Comments on what can be inferred from the low confidence-accuracy correlation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(5), 1304-1316.
data(lineup_example)
cac <- make_cacdata(lineup_example, confidence_bins = c(0, 60, 80, 100))
cac$cac_data
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.