wevid-package: Quantifying performance of a diagnostic test using the...
In wevid: Quantifying Performance of a Binary Classifier Through Weight of Evidence

Description Details Author(s) References See Also

The wevid package provides functions for quantifying the performance of a diagnostic test (or any other binary classifier) by calculating and plotting the distributions in cases and noncases of the weight of evidence favouring case over noncase status.

The distributions of the weight of evidence (log Bayes factor) favouring case over noncase status in a test dataset (or test folds generated by cross-validation) can be used to quantify the performance of a diagnostic test.

In comparison with the C-statistic (area under ROC curve), the expected weight of evidence (expected information for discrimination) has several advantages as a summary measure of predictive performance. To quantify how the predictor will behave as a risk stratifier, the quantiles of the distributions of weight of evidence in cases and controls can be calculated and plotted.

This package can be used with any test dataset on which you have observed case-control status and have computed prior and posterior probabilities of case status using a model learned on a training dataset. Therefore, you should have computed on a test dataset (or on test folds used for cross-validation):

The prior probability of case status (this may be just the frequency of cases in the training data).
The posterior probability of case status (using the model learned on the training data to predict on the test data).
The observed case status (coded as 0 for noncases, 1 for cases).

The main function of the package is Wdensities which computes the crude and model-based densities of weight of evidence in cases and controls. Once these are computed, they can be plotted with plotWdists and plotcumfreqs. Summary statistics can be reported with summary.

Paul McKeigue paul.mckeigue@ed.ac.uk

Paul McKeigue (2019), Quantifying performance of a diagnostic test as the expected information for discrimination: Relation to the C-statistic. Statistical Methods for Medical Research, 28 (6), 1841-1851. https://doi.org/10.1177/0962280218776989.

Useful links: