metrics: Performance Metrics

In MRMCaov: Multi-Reader Multi-Case Analysis of Variance

Performance Metrics

Description

Estimated performance metrics from ROC curves.

Usage

binary_sens(truth, rating)

binary_spec(truth, rating)

binormal_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

binormal_eu(truth, rating, slope = 1)

binormal_sens(truth, rating, spec)

binormal_spec(truth, rating, sens)

binormalLR_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

binormalLR_eu(truth, rating, slope = 1)

binormalLR_sens(truth, rating, spec)

binormalLR_spec(truth, rating, sens)

empirical_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

empirical_eu(truth, rating, slope = 1)

empirical_sens(truth, rating, spec)

empirical_spec(truth, rating, sens)

trapezoidal_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

trapezoidal_sens(truth, rating, spec)

trapezoidal_spec(truth, rating, sens)

Arguments

truth	vector of true binary statuses.
rating	vector of 0-1 binary ratings for the binary metrics and ranges of numeric ratings for the others.
partial	character string "sensitivity" or "specificity" for calculation of partial AUC, or FALSE for full AUC. Partial matching of the character strings is allowed. "specificity" results in area under the ROC curve between the given min and max specificity values, whereas "sensitivity" results in area to the right of the curve between the given sensitivity values.
min, max	minimum and maximum sensitivity or specificity values over which to calculate partial AUC.
normalize	logical indicating whether partial AUC is divided by the interval width (max - min) over which it is calculated.
slope	slope of the iso-utility line at which to compute expected utility of the ROC curve.
sens, spec	numeric sensitivity/specificity at which to calculate specificity/sensitivity.

Details

Performance metrics measure the degree to which higher case ratings are associated with positive case statuses, where positive status is taken to be the highest level of truth. Available metrics include area under the ROC curve (auc), expected utility of the ROC curve (eu) at a given iso-utility line (Abbey, 2013), sensitivity (sens) at a given specificity, and specificity (spec) at a given sensitivity.

Value

Returns a numeric value.

References

Abbey CK, Samuelson FW and Gallas BD (2013). Statistical power considerations for a utility endpoint in observer performance studies. Academic Radiology, 20: 798-806.

See Also

mrmc, srmc, stmc

MRMCaov documentation built on Jan. 11, 2023, 1:14 a.m.