doIMRMC: MRMC analysis for arbitrary design dataset

In iMRMC: Multi-Reader, Multi-Case Analysis Methods (ROC, Agreement, and Other Metrics)

MRMC analysis for arbitrary design dataset

Description

Execute a Multi-Reader, Multi-Case (MRMC) analysis of ROC data from imaging studies where clinicians (readers) evaluate patient images (cases). This function is a wrapper that executes doAUCmrmc and formats the output to generally match the output of doIMRMC version 1.2.5. An overview of this software, including references presenting details on the methods, can be found HERE or as an entry in the FDA/CDRH Regulatory Science Tool Catalog HERE.

Usage

doIMRMC(data)

Arguments

data	an iMRMC formatted data frame, see dfMRMC_example

Details

Unlike the legacy doIMRMC_java, the 'varDecomp' results no longer scale the covariance by a factor of 2. This scaling is needed when calculating the total variance of the difference in modalities. The user must scale this covariance by 2 manually now to achieve the total variance of the difference in modalities result.

Value

The MRMC analysis results, below is a quick summary:

• perReader, data.frame The performance results for each combination of reader and pair of modalities. Key variables of this data frame are AUCA, AUCB, AUCAminusAUCB and the corresponding variances. When the modalities differ, the variance is understood to be the covariance between the modalities.

• Ustat, data.frame Reader-averaged performance results for each pair of modalities. The analysis results are based on U-statistics. Key variables of this data frame are AUCA, AUCB, AUCAminusAUCB and the corresponding variances, confidence intervals, degrees of freedom, and p-values. When the modalities differ, the variance is understood to be the covariance between the modalities.

• MLEstat, data.frame Reader-average performance results for each pair of modalities. The analysis results are based on V-statistics, which approximates the true distribution with the empirical distribution. The empirical distribution equals the nonparametric MLE estimate of the true distribution, which is also equivalent to the ideal bootstrap estimate. Key variables of this data frame are AUCA, AUCB, AUCAminusAUCB and the corresponding variances, confidence intervals, degrees of freedom, and p-values. When the modalities differ, the variance is understood to be the covariance between the modalities.

• varDecomp, list list of data frames of the coefficient and components of variance. The analysis includes variance decomposition based off both the BDG and BCK MRMC methods, and Ustat and MLE statistical methods. Each MRMC and statistical method combination is contained within this list of lists.

• ROC, list each object of this list is an object containing an ROC curve. There is an ROC curve for every combination of reader and modality. For every modality, there are also four average ROC curves. These are discussed in Chen2014_Br-J-Radiol_v87p20140016. The diagonal average averages the reader-specific ROC curves along y = -x + b for b in (0,1). The horizontal average averages the reader specific ROC curves along y = b for b in (0,1). The vertical average averages the reader specific ROC curves along x = b for b in (0,1). The pooled average ignores readerID and pools all the scores together to create one ROC curve.

• full, list This returns the same result as doAUCmrmc.

Examples

# Create a sample configuration file
config <- sim.gRoeMetz.config()
# Simulate an MRMC ROC data set
dFrame.imrmc <- sim.gRoeMetz(config)
# Analyze the MRMC ROC data
result <- doIMRMC(dFrame.imrmc)

iMRMC documentation built on Sept. 11, 2024, 7:12 p.m.