uStat11: Analysis of U-statistics degree 1,1

In iMRMC: Multi-Reader, Multi-Case Analysis Methods (ROC, Agreement, and Other Metrics)

Analysis of U-statistics degree 1,1

Description

These two functions calculate the mean and variance of a user-specified U-statistic kernel, which is a function of cross-correlated scores.

The motivation for this analysis is data collected in imaging studies where multiple readers read multiple cases in different modes or modalities. The goal is to evaluate the variance of a reader- and case-averaged endpoint, accounting for cross-correlated data arising from two random effects: the random reader skill and the random case difficulty. This analysis is sometimes referred to as an MRMC analysis. Of course, the random effects can be from sources other than readers and cases.

Usage

uStat11.jointD(
  df.input,
  modalitiesToCompare,
  kernelFlag = 1,
  keyColumns = c("readerID", "caseID", "modalityID", "score")
)

uStat11.conditionalD(
  df.input,
  modalitiesToCompare,
  kernelFlag = 1,
  keyColumns = c("readerID", "caseID", "modalityID", "score")
)

Arguments

df.input	Data frame of observations, one per row. Columns identify random effects, fixed effects, and the observation. Namely, readerID: The factor corresponding to the different readers in the study. The readerID is treated as a random effect. caseID: The factor corresponding to the different cases in the study. The caseID is treated as a random effect. modalityID: The factor corresponding to the different modalities in the study. The modalityID is treated as a fixed effect. score: The score given by the reader to the case for the modality indicated.
modalitiesToCompare	The factors identifying the modalities to compare.
kernelFlag	This determines the kernel function kernelFlag = 1 == identity kernel: requires two modalities: A,B. kernelFlag = 2 == kernel of the difference in modalities: requires four modalities: A,B,C,D.
keyColumns	Identify the factors corresponding to the readerID, caseID, modalityID, and score (or alternative random and fixed effects).

Details

uStat11.conditionalD is identical to uStat11.jointD when the study is fully-crossed: when every reader readers all the cases in both modalities. For arbitrary study designs the two functions differ according to how the components of variance are estimated.

• uStat11.conditionalD follows Gallas2007_J-Opt-Soc-Am-A_v24pB70 <doi:10.1364/JOSAA.24.000B70> and estimates the components of variance (which isolate combinations of different random effects) with nested conditional means.

• uStat11.jointD is analogous to the method in Gallas2008_Neural-Networks_v21p387 <doi:10.1016/j.neunet.2007.12.013> and estimates the components of variance (which isolate combinations of different random effects) with a joint distribution over all the observations giving equal weight to each one.

Both functions yield unbiased variance estimates. Our simulations find that uStat11.conditionalD is statistically more efficient than uStat11.jointD (its variance estimate is more precise), but it is slower.

Please refer to the tests/testthat folder of the package for examples using these functions.

Value

This function calculates the mean and variance of the indicated U-statistic kernel, which is a function of the scores. For the identity kernel, we simply return the mean and variance of the scores.

The function returns a list of outputs. Many of these outputs have three elements.

• If kernelFlag = 1 == identity kernel, the first element corresponds to the mean score of modality A, the second corresponds to mean score of modality B, and the third corresponds to the mean of the difference in scores from modality A and B.

• If kernelFlag = 2 == difference kernel, the first element corresponds to the mean difference in scores from modalities A and B, the second element corresponds to the mean difference in scores from modalities C and D, and the third elements corresponds to the difference of the just-mentioned differences.

There are 16 outputs:

• mean: See description above.

• var: The variance of the mean.

• var.1obs: The variance of one reader-case-modality observation.

• meanPerR The reader-specific means.

• nR The number of readers in the study.

• nC The number of cases in the study.

• nCperR The number of cases evaluated by each reader for each modality.

• moments The second order moments of the problem.

• coeff The coefficients corresponding to the second-order moments such that the scalar product between the moments and coefficients yields the variance.

• kernel.A A matrix showing the kernel evaluated for each combination of each reader and case for modality A (or AB).

• design.A A matrix showing the what data exists for each combination of each reader and case for modality A (or AB).

• kernel.B A matrix showing the kernel evaluated for each combination of each reader and case for modality B (or CD).

• design.B A matrix showing the what data exists for each combination of each reader and case for modality B (or CD).

Examples

# Create an MRMC data frame
# Refer to Gallas2014_J-Med-Img_v1p031006
simRoeMetz.config <- sim.gRoeMetz.config()

# Simulate data
df.MRMC <- sim.gRoeMetz(simRoeMetz.config)

# Reformat data
df <- undoIMRMCdf(df.MRMC)

# Grab part of the data
df <- droplevels(df[grepl("pos", df$caseID), ])

#### uStat11.jointD.identity ####
# Calculate the reader- and case-averaged difference in scores from testA and testB
# (kernelFlag = 1 specifies the U-statistics kernel to be the identity)
result.jointD.identity <- uStat11.jointD(
  df,
  kernelFlag = 1,
  keyColumns = c("readerID", "caseID", "modalityID", "score"),
  modalitiesToCompare = c("testA", "testB"))

cat("\n")
cat("uStat11.jointD.identity \n")
print(result.jointD.identity[1:2])

iMRMC documentation built on May 31, 2023, 8:36 p.m.