OR_to_RMH | R Documentation |
Determines Roe & Metz (RM) simulation model parameters for simulating multireader multicase likelihood-of-disease rating data based on real-data or conjectured Obuchowski-Rockette (OR) parameter estimates that describe the distribution of the empirical AUC reader performance measure. The algorithm assumes the constrained unequal-variance RM model (Hillis, 2012) which generalizes the original RM model (Roe and Metz, 1997) by allowing the diseased and nondiseased decision-variable distributions to have unequal variances for each reader, with the variance components involving diseased cases constrained to differ by a factor of 1/b^2 from corresponding variance components involving nondiseased cases. This algorithm is described in Hillis (2020). Throughout we refer to the Hillis (2012) RM model as the RMH model.
OR_to_RMH(...) ## Default S3 method: OR_to_RMH( ..., AUC1, AUC2, var_R, var_TR, corr1, corr2, corr3, var_error = NULL, n0, n1, b_method = c("unspecified", "mean_to_sigma", "specified"), mean_sig_input = NULL, b_input = NULL, b_le_1 = TRUE ) ## S3 method for class 'data.frame' OR_to_RMH(params, ...)
... |
arguments passed to the default method. |
AUC1, AUC2 |
test 1 and 2 expected empirical AUCs. |
var_R, var_TR |
OR reader and test-by-reader variance components. |
corr1, corr2, corr3 |
OR error correlations. |
var_error |
OR error variance. |
n0, n1 |
number of nondiseased and diseased cases. |
b_method |
method of estimating RMH b parameter. |
mean_sig_input |
mean-to-sigma ratio, required only if
|
b_input |
binormal b value, required only if
|
b_le_1 |
logical indicating whether the algorithm searches first for
b <= 1 and then, if no solution, for b >= 1; if FALSE, the algorithm
searches only for for b >= 1. Required only if
|
params |
data frame of above OR parameter values in the columns. |
Hillis (2012) modified the original RM model (Roe and Metz, 1997) by allowing variance components involving case to depend on truth (diseased/nondiseased), with variance components involving diseased cases set equal to those involving nondiseased cases multiplied by the factor 1/b^2, b>0. We refer to the Hillis (2012) model as the RMH model. Hillis (2018) derived analytical formulas that express OR parameters describing the distribution of empirical AUC outcomes computed from RMH simulated data as functions of the RMH model parameters. The reverse mapping from the RMH parameters to the OR parameters is implemented in R by the RMH_to_OR function. The OR_to_RMH function uses an iterative search procedure.
b_method
indicates the method for estimating the RMH b
parameter. Note that b is the conventional binormal-curve slope, i.e.,
the slope of each reader's true ROC curve plotted in probit space.
b_method = "unspecified"
should be used when the goal is to
determine RM parameters that result in simulated data for which the
empirical AUC distribution is described by the inputted values for the
OR parameter vector
β_OR = (AUC1, AUC2, var_R, var_TR, var_error, corr1, corr2, corr3).
b_method = "mean_to_sigma"
should be used when the goal is to
determine RM parameters that result in simulated data for which the
empirical AUC distribution is described by the inputted values for the OR
parameter vector
β1_OR = (AUC1, AUC2, var_R, var_TR, corr1, corr2, corr3),
and such that the median mean-to-sigma ratio across
readers is equal to mean_sig_input
for the test having the lowest
AUC. Note that β1_OR differs from β_OR in that it
does not contain the OR error variance.
b_method = "specified"
should be used when the goal is to
determine RM parameters that result in simulated data for which the
empirical AUC distribution is described by the inputted values for the OR
parameter vector β1_OR (see above) with b equal to
mean_sig_input
. (E.g., set b_input = 1
for symmetric
ROC curve.)
For b_method = "mean_to_sigma"
or "specified"
, the simulated
empirical AUC estimate distribution is specified by the parameter values
in params
, except for var_error
. Thus for these two options,
var_error
can be equal to NA
or excluded from params
.
Parameter mean_sig_input
is the inputted mean-to-sigma ratio needed
for b_method = "mean_to_sig"
. See Hillis & Berbaum (2011) for more
information.
Parameter b_input
is the inputted binormal b value needed for
b_method = "specified"
.
There may not be a solution for a set of OR parameters values. When this occurs, the function will either produce an approximate solution or indicate what OR input needs to be changed.
A related function is the RMH_to_OR function, which determines OR parameters that describe the distribution of empirical AUC estimates computed from inputted RM model parameter values, based on the analytical mapping provided by Hillis (2018).
The RMH model parameters are returned in a data frame with the following elements.
mean separation of nondiseased and diseased decision-variable distributions for test 1 across reader population.
mean separation of nondiseased and diseased decision-variable distributions for test 2 across reader population.
RMH reader variance compnent.
RMH text-by-reader variance component.
RMH case variance component.
RMH test-by-case variance.
RMH reader-by-case variance.
RMH error variance.
variance components involving diseased cases are constrained to differ by a factor of 1/b^2 from corresponding variance components involving nondiseased cases.
Related quantities that are also returned in the data frame:
method used to estimate b.
number of nondiseased cases per simulated sample.
number of diseased cases per simulated sample.
expected mean-to-sigma ratio across readers for test 1.
expected mean-to-sigma ratio across readers for test 2.
probability that the test 1 ROC curve for a random reader will be noticeably improper (i.e, |mean-to-sigma ratio| < 2).
probability that the test 2 ROC curve for a random reader will be noticeably improper (i.e, |mean-to-sigma ratio| < 2).
Stephen L. Hillis, Departments of Radiology and Biostatistics, University of Iowa, steve-hillis@uiowa.edu
Brian J. Smith, Department of Biostatistics, University of Iowa, brian-j-smith@uiowa.edu
Hillis SL (2012). Simulation of unequal-variance binormal multireader ROC decision data: an extension of the Roe and Metz simulation model. Academic Radiology, 19(12): 1518-1528. doi: 10.1016/j.acra.2012.09.011
Hillis SL (2018). Relationship between Roe and Metz simulation model for multireader diagnostic data and Obuchowski-Rockette model parameters. Statistics in Medicine, 37(13): 2067-2093. doi: 10.1002/sim.7616
Hillis SL (2020). Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on read-data or conjectured Obuchowski-Rockette parameter estimates. Vol. 11316, SPIE Medical Imaging: SPIE. doi.org/10.1117/12.2550541
Hillis SL and Kevin SB (2011). Using the mean-to-sigma ratio as a measure of the improperness of binormal ROC curves. Academic Radiology, 18(2): 143-154. doi: 10.1016/j.acra.2010.09.002
Roe CA and Metz CE (1997). Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic data: validation with computer simulation. Academic Radiology, 4(4): 298-303. doi: 10.1016/S1076-6332(97)80032-3
RMH_to_OR
## Example 1: Computing RM parameters from OR parameters directly ##-------------------------------------------------------------- ## Example 1a: Using b_method ="unspecified" (the default) RM <- OR_to_RMH(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, var_error = 0.000788) RM ## We recommend also computing the OR parameter values ("true values") # that describe the distribution of simulated data based on above RM parameters, # using the RMH_to_OR function. Ideally the true values will be the same as the # inputted OR values used for deriving the RM parameter values. We recommend # always performing this check. This check is carried out below, as shown below. true_values = RMH_to_OR(RM) true_values # From the output we see, for this example, that the true OR values are identical to the # inputted OR values # Example 1b: Using b_method = "specified" with b_input = 1 # Note that the error variance does not need to be specified since this b_method # does not utilize it. RM <- OR_to_RMH(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, b_method = "specified", b_input = 1) RM true_values <- RMH_to_OR(RM) true_values # From the output we see, for this example, that the true values are identical # (within rounding error) to the inputted OR values (but note that var_error was # not inputted) ## Example 1c: Using b_method = "mean_to_sigma" with mean_to_sig_input = 4.5 # Note the error variance does not need to be specified since this b_method # does not utilize it. RM <- OR_to_RMH(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, b_method = "mean_to_sigma", mean_sig_input = 4.5) RM true_values <- RMH_to_OR(RM) true_values # From the output we see for this example that the true OR values are identical # (within rounding error) to the inputted OR values (but note that var_error was # not inputted) ##--------------------------------------------------------------------- ## Example 2: Computing RM parameters from a data frame of OR parameters ## --------------------------------------------------------------------- ## Example 2a: One study vandyke_OR <- data.frame(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, var_error = 0.000788) vandyke_RM <- OR_to_RMH(vandyke_OR) vandyke_RM true_values <- RMH_to_OR(vandyke_RM) true_values ## Example 2b: Three studies three_studies_OR <- data.frame( rbind( vandyke = c(69, 45, 0.89793704, 0.94083736, 0.432, 0.429, 0.298, 0.00154, 0.0002, 0.00080229), franken = c(33, 67, .8477498869, 0.8368950701, 0.521430051, 0.319691199, 0.3386375697, 0.0000433385, 0.0, 0.0014967254), kundel = c(66, 29, 0.8038793103, 0.8413662487, 0.507695244, 0.3843523762, 0.4035662578, 0.0007340122, 0, 0.002148844) ) ) colnames(three_studies_OR) <- c("n0", "n1", "AUC1", "AUC2", "corr1", "corr2", "corr3", "var_R", "var_TR", "var_error") three_studies_OR three_studies_RM <- OR_to_RMH(three_studies_OR) three_studies_RM true_values <- RMH_to_OR(three_studies_RM) true_values ## Note above that the true values for corr2 and corr3 for the Franken study # differ slightly from the inputted values; this is because corr2 < corr3 for the # inputted OR values, which is not possible for simulated RM model data. ##Example 2c: Examples 1a, 1b and 1c run using one data frame vandyke_OR <- data.frame(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, var_error = 0.000788) vandyke_OR_x3 <- vandyke_OR[c(1,1,1),] #has 3 rows, each equal to vandyke_OR b_method = c("unspecified","mean_to_sigma","specified") mean_sig_input = c(NA,4.5,NA) b_input = c(NA,NA,1) vandyke_OR_3ex <- cbind(vandyke_OR_x3,b_method,mean_sig_input,b_input) vandyke_OR_3ex vandyke_OR_3ex_RM <- OR_to_RMH(vandyke_OR_3ex) vandyke_OR_3ex_RM true_values <- RMH_to_OR(vandyke_OR_3ex_RM) true_values ## Example 3: Printing the alternative x1 -- x7 parameter values ## --------------------------------------------------- ## The OR_to_RMH function first finds the solutions using the alternative RM ## parameterization consisting of b and the alternative parameters ## x1, x2, x3, x4, x5, x6, and x7, and then solves for the conventional RM ## parameters in terms of these alternative parameters. (See Hillis (2020) for details.) ## Although the user generally has no need to know these parameter values, they ## can be printed out using the all = TRUE print option, as shown below ## using Example 1a: RM <- OR_to_RMH(n0 = 69, n1 = 45, AUC1 = 0.897, AUC2 = 0.941, corr1 = 0.433, corr2 = 0.430, corr3 = 0.299, var_R = 0.00154, var_TR = 0.000208, var_error = 0.000788) print(RM,all = TRUE)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.