UtilFigureOfMerit: Calculate empirical figures of merit (FOMs) for specified...

In RJafroc: Artificial Intelligence Systems and Observer Performance

Calculate empirical figures of merit (FOMs) for specified dataset

Description

Calculate the specified empirical figure of merit for each treatment-reader combination in the ROC, FROC, ROI or LROC dataset

Usage

UtilFigureOfMerit(dataset, FOM = "wAFROC", FPFValue = 0.2)

Arguments

dataset	The dataset to be analyzed, RJafroc-package
FOM	The figure of merit; the default is "wAFROC"
FPFValue	Only needed for LROC data and FOM = "PCL" or "ALROC"; where to evaluate a partial curve based figure of merit. The default is 0.2.

Details

The allowed FOMs depend on the dataType field of the dataset object.

For dataset$descriptions$design = "SPLIT-PLOT-C", end-point based FOMs (e.g., "MaxLLF") are not allowed. For dataset$descriptions$type = "ROC" only FOM = "Wilcoxon" is allowed. For dataset$descriptions$type = "FROC" the following FOMs are allowed:

• FOM = "AFROC1" (use only if zero normal cases)

• FOM = "AFROC"

• FOM = "wAFROC1" (use only if zero normal cases)

• FOM = "wAFROC" (the default)

• FOM = "HrAuc"

• FOM = "SongA1"

• FOM = "SongA2"

• FOM = "HrSe" (an example of an end-point based FOM)

• FOM = "HrSp" (another example)

• FOM = "MaxLLF" (do:)

• FOM = "MaxNLF" (do:)

• FOM = "MaxNLFAllCases" (do:)

• FOM = "ExpTrnsfmSp"

"MaxLLF", "MaxNLF" and "MaxNLFAllCases" correspond to ordinate, and abscissa, respectively, of the highest point on the FROC operating characteristic obtained by counting all the marks. The "ExpTrnsfmSp" FOM is described in the paper by Popescu. Given the large number of FOMs possible with FROC data, it is appropriate to make a recommendation: it is recommended that one use the wAFROC FOM whenever possible.

For dataType = "ROI" dataset only FOM = "ROI" is allowed.

For dataType = "LROC" dataset the following FOMs are allowed:

• FOM = "Wilcoxon" for ROC data inferred from LROC data

• FOM = "PCL" the probability of correct localization at specified FPFValue

• FOM = "ALROC" the area under the LROC from zero to specified FPFValue

FPFValue The FPF at which to evaluate PCL or ALROC; the default is 0.2; only needed for LROC data.

Value

An c(I, J) dataframe, where the row names are modalityID's of the treatments and column names are the readerID's of the readers.

References

Chakraborty DP (2017) Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, CRC Press, Boca Raton, FL. https://www.routledge.com/Observer-Performance-Methods-for-Diagnostic-Imaging-Foundations-Modeling/Chakraborty/p/book/9781482214840

Chakraborty DP, Berbaum KS (2004) Observer studies involving detection and localization: modeling, analysis, and validation, Medical Physics, 31(8), 1–18.

Song T, Bandos AI, Rockette HE, Gur D (2008) On comparing methods for discriminating between actually negative and actually positive subjects with FROC type data, Medical Physics 35 1547–1558.

Popescu LM (2011) Nonparametric signal detectability evaluation using an exponential transformation of the FROC curve, Medical Physics, 38(10), 5690.

Obuchowski NA, Lieber ML, Powell KA (2000) Data Analysis for Detection and Localization of Multiple Abnormalities with Application to Mammography, Acad Radiol, 7:7 553–554.

Swensson RG (1996) Unified measurement of observer performance in detecting and localizing target objects on images, Med Phys 23:10, 1709–1725.

Examples

UtilFigureOfMerit(dataset02, FOM = "Wilcoxon") # ROC data
UtilFigureOfMerit(DfFroc2Roc(dataset01), FOM = "Wilcoxon") # FROC dataset, converted to ROC
UtilFigureOfMerit(dataset01) # FROC dataset, default wAFROC FOM
UtilFigureOfMerit(datasetCadLroc, FOM = "Wilcoxon") #LROC data
UtilFigureOfMerit(datasetCadLroc, FOM = "PCL") #LROC data
UtilFigureOfMerit(datasetCadLroc, FOM = "ALROC") #LROC data
UtilFigureOfMerit(datasetROI, FOM = "ROI") #ROI data
 # these are meant to illustrate conditions which will throw an error
## UtilFigureOfMerit(dataset02, FOM = "wAFROC") #error
## UtilFigureOfMerit(dataset01, FOM = "Wilcoxon") #error

RJafroc documentation built on Nov. 10, 2022, 5:45 p.m.