View source: R/model_comparison.R
| compare_models | R Documentation |
Fits and compares multiple models (2-HT, EIG, Full ROC) to the same lineup identification dataset, providing a side-by-side descriptive table.
compare_models(
data,
models = c("2ht", "eig", "fullroc"),
lineup_size = 6,
prior_guilt = 0.5,
confidence_bins = NULL,
show_warnings = FALSE,
...
)
data |
A dataframe with columns: target_present, identification, confidence |
models |
Character vector of models to fit. Options:
Default = c("2ht", "eig", "fullroc") fits all models. |
lineup_size |
Integer. Number of people in the lineup (default = 6) |
prior_guilt |
Numeric. Prior probability of guilt for EIG (default = 0.5) |
confidence_bins |
Numeric vector of confidence bin edges (optional). Used for EIG and Full ROC if specified. |
show_warnings |
Logical. Whether to show model fitting warnings (default = FALSE) |
... |
Additional arguments passed to individual model functions |
This function provides a unified interface for fitting and comparing multiple eyewitness identification models. It automatically handles different data requirements and output formats across models.
**Models Compared:**
**2-HT (Winter et al., 2022)**: Multinomial processing tree model with parameters for detection (dP, dA), bias (b), and guessing (g). Provides AIC/BIC for comparison only with other likelihood models fit to the same observations and outcome representation.
**EIG (Starns et al., 2023)**: Information-theoretic measure of evidentiary value. Higher values indicate more diagnostic procedures.
**Full ROC (Smith et al., 2020)**: Uses ALL responses to compute investigator discriminability. AUC ranges from 0 to 1; 0.5 represents chance under a fixed ordering.
**Interpretation:**
AIC/BIC describe the 2-HT likelihood fit but cannot be compared with EIG or full-ROC AUC
Use EIG for comparing procedure diagnosticity (higher is better,
at the same prior_guilt and with the same binning of responses)
Use Full ROC AUC for investigator discriminability (higher is better; the default diagnosticity ordering is optimistic, so compare procedures with the a-priori order)
An object of class "model_comparison" containing:
comparison_table: Dataframe comparing model fits
fitted_models: List of fitted model objects
best_model: Always NULL. The included methods estimate different quantities and cannot be ranked by a common information criterion.
data: The input data
models_fit: Character vector of models successfully fit
Winter, K., Menne, N. M., Bell, R., & Buchner, A. (2022). Experimental validation of a multinomial processing tree model for analyzing eyewitness identification decisions. Scientific Reports, 12, 15571.
Starns, J. J., Cohen, A. L., & Rotello, C. M. (2023). A complete method for assessing the effectiveness of eyewitness identification procedures: Expected information gain. Psychological Review, 130(3), 677–719.
Smith, A. M., Yang, Y., & Wells, G. L. (2020). Distinguishing between investigator discriminability and eyewitness discriminability. Perspectives on Psychological Science, 15(3), 589-607.
data(lineup_example)
# Fit all models
comparison <- compare_models(lineup_example)
print(comparison)
summary(comparison)
# Fit specific models
comparison <- compare_models(lineup_example, models = c("2ht", "eig"))
# Access individual fitted models
comparison$fitted_models$`2ht`
comparison$fitted_models$eig
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.