View source: R/dpp_functions.R
| make_dpp | R Documentation |
Computes the average deviation from perfect performance of Smith, Lampinen, Wells, Smalarz and Mackovichova (2019). DPP is calculated at every operating point (cumulative confidence threshold) of the ROC and averaged over them, so that, unlike pAUC, it is not tied to a region of false-ID rates that two differently truncated ROC curves may not share.
make_dpp(
data,
lineup_size = 6,
use_roc_obj = FALSE,
alpha = NULL,
beta = NULL,
innocent_suspect = "auto",
thresholds = NULL
)
data |
A dataframe with the following columns:
Alternatively, with |
lineup_size |
Integer. Number of people in lineup (default = 6) |
use_roc_obj |
Logical. If FALSE (default), computes ROC from data.
If TRUE, data should be the output from make_rocdata(), whose first
|
alpha, beta |
Optional weights for Smith et al.'s Equation 3.
|
innocent_suspect |
How innocent-suspect identifications in
target-absent lineups are obtained. |
thresholds |
Optional numeric vector of confidence criteria at which to evaluate the ROC. The default uses the confidence levels of the responses that can move an operating point; see Details. Supplying the criteria of a full sample keeps them fixed across bootstrap resamples. |
At an operating point c, Smith et al.'s (2019) Equation 2 is
DPP(c) = 1 - Culprit(c) + Innocent(c)
where Culprit(c) is the culprit (correct) identification rate and
Innocent(c) the innocent-suspect identification rate at that point.
DPP(c) ranges from 0 (perfect performance: every culprit identified, no
innocent suspect identified) to 2 (the reverse). Any point on the chance
diagonal has DPP(c) = 1. When the base rate of culprit presence is .5 and
the costs of the two errors are equal, DPP(c) = 1 - U(c), where
U(c) = Culprit(c) - Innocent(c) is the expected utility.
The reported dpp is the mean of DPP(c) over the operating points,
one for each confidence criterion of the ROC, from the highest to the
lowest (which includes every identification). Lower values are better. The
criteria are the confidence levels of the responses that enter the ROC
(see make_rocdata), so the confidence given to rejections
does not add points or change the average.
Equation 3 weights the two errors:
DPP_{\alpha\beta}(c) = \beta[1 - Culprit(c)] + \alpha(1 - \beta)Innocent(c)
With alpha = 1 and beta = 0.5 this is half of Equation 2.
Weighted values no longer range from 0 to 2.
A list containing:
dpp: Average DPP over the operating points
dpp_by_point: Tibble with confidence threshold, culprit (correct-ID) rate, innocent-suspect ID rate and DPP at each operating point
best_point: The operating point with the lowest DPP, which is the point of maximum utility
roc_data: ROC curve data from make_rocdata()
alpha, beta: The weights used (NULL for Equation 2)
n_target_present, n_target_absent: Sample sizes
innocent_suspect_method: How innocent-suspect IDs were obtained
thresholds: The confidence criteria of the operating points
Smith, A. M., Lampinen, J. M., Wells, G. L., Smalarz, L., & Mackovichova, S. (2019). Deviation from perfect performance measures the diagnostic utility of eyewitness lineups but partial area under the ROC curve does not. Journal of Applied Research in Memory and Cognition, 8(1), 50-59. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.jarmac.2018.09.003")}
data(lineup_example)
dpp <- make_dpp(lineup_example)
dpp$dpp
dpp$dpp_by_point
# Innocent-suspect IDs twice as costly as misses, culprit base rate .5
make_dpp(lineup_example, alpha = 2, beta = 0.5)$dpp
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.