View source: R/utility_functions.R
| make_utility_curves | R Documentation |
Computes expected utility at different confidence criteria following Lampinen, Smith, & Wells (2019). Utility analysis accounts for the costs and benefits of different identification outcomes.
make_utility_curves(
data,
base_rate = 0.5,
utility_matrix = c(tp = 1, fn = -0.5, fp = -2, tn = 0.5),
lineup_size = 6,
criteria = c("confidence", "all"),
innocent_suspect = "auto",
thresholds = NULL
)
data |
A dataframe with the following columns:
|
base_rate |
Numeric. Prior probability that suspect is guilty (0 to 1). Default = 0.5. |
utility_matrix |
Named numeric vector with utilities/costs for:
Default: c(tp=1, fn=-0.5, fp=-2, tn=0.5).
Only the differences |
lineup_size |
Integer. Number of people in lineup (default = 6). Used to estimate false IDs from filler choices. |
criteria |
Character. How to define decision criteria:
Default = "confidence". |
innocent_suspect |
How innocent-suspect identifications in
target-absent lineups are obtained. |
thresholds |
Optional numeric vector of confidence criteria. The
default uses the confidence levels of the responses that can change the
hit or false-alarm rate, as in |
Expected utility combines hit rates and false alarm rates with the costs/benefits of different outcomes:
EU = base\_rate \cdot [hit \cdot U_{TP} + (1-hit) \cdot U_{FN}] +
(1-base\_rate) \cdot [fa \cdot U_{FP} + (1-fa) \cdot U_{TN}]
Where:
hit = p(suspect ID | guilty) at criterion
fa = p(suspect ID | innocent) at criterion
U_TP, U_FN, U_FP, U_TN = utilities for each outcome
The utility matrix should reflect the relative value/cost of outcomes. For example:
tp = 1: Correctly identifying the perpetrator
fn = -0.5: Missing the perpetrator (they remain at large)
fp = -2: Wrongly convicting an innocent person (severe injustice)
tn = 0.5: Correctly rejecting when innocent
Lampinen et al. (2019) show that comparing procedures by ROC curves alone can be misleading. Utility analysis incorporates base rates and the relative costs of errors, providing a more complete evaluation.
Note on the false-alarm estimate: explicit target-absent suspect IDs are used when present. Otherwise, filler IDs are divided by lineup size. Direct and filler-derived estimates are never added together.
A list containing:
utility_data: Dataframe with criterion, strategy, hit rate,
false alarm rate, and expected utility. Each row is the rule "accept
suspect IDs with confidence >= criterion". The "reject_all"
row has a criterion above the observed identification confidences
and supplied grid (possibly Inf), so that
applying it accepts no identification.
max_utility: Maximum expected utility, with the corresponding criterion and strategy
avg_utility: Average expected utility across all criteria
utility_all_ids: Expected utility if all IDs are accepted
base_rate: Base rate used
utility_matrix: Utility matrix used
Lampinen, J. M., Smith, A. M., & Wells, G. L. (2019). Four utilities in eyewitness identification practice: Dissociations between receiver operating characteristic analysis and expected utility analysis. Law and Human Behavior, 43(1), 26-44.
data(lineup_example)
util <- make_utility_curves(lineup_example)
util$max_utility
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.