StSignificanceTesting: Performs DBM or OR significance testing for factorial or...

In RJafroc: Artificial Intelligence Systems and Observer Performance

Performs DBM or OR significance testing for factorial or split-plot A,C datasets

Description

Performs Dorfman-Berbaum-Metz (DBM) or Obuchowski-Rockette (OR) significance testing, for specified dataset; significance testing refers to analysis designed to assign a P-value, and other statistics, for rejecting the null hypothesis (NH) that the reader-averaged figure of merit (FOM) differences between treatments is zero. The results of the analysis are best visualized in the text or Excel-formatted files produced by UtilOutputReport.

Usage

StSignificanceTesting(
  dataset,
  FOM,
  FPFValue = 0.2,
  alpha = 0.05,
  method = "DBM",
  covEstMethod = "jackknife",
  nBoots = 200,
  analysisOption = "ALL",
  tempOrgCode = FALSE
)

Arguments

dataset	The dataset to be analyzed, see RJafroc-package. Must have two or more treatments and two or more readers. The dataset design can be "FCTRL", "SPLIT-PLOT-A" or "SPLIT-PLOT-C".
FOM	The figure of merit, see UtilFigureOfMerit
FPFValue	Only needed for LROC data and FOM = "PCL" or "ALROC"; where to evaluate a partial curve based figure of merit. The default is 0.2.
alpha	The significance level of the test of the null hypothesis that all treatment effects are zero; the default is 0.05
method	The significance testing method to be used: "DBM" (the default), representing the Dorfman-Berbaum-Metz method or "OR", representing the Obuchowski-Rockette method. and the Obuchowski-Rockette significance testing methods, respectively.
covEstMethod	The covariance matrix estimation method in ORH analysis (for method = "DBM" the jackknife is always used). "Jackknife", the default, "Bootstrap", in which case nBoots (above) is relevant, "DeLong"; requires FOM = "Wilcoxon" or "ROI" or "HrAuc", otherwise an error results.
nBoots	The number of bootstraps (defaults to 200), relevant only if covEstMethod = "bootstrap" and method = "OR"
analysisOption	Determines which factors are regarded as random vs. fixed: "RRRC" = random-reader random case, "FRRC" = fixed-reader random case, "RRFC" = random-reader fixed case, "ALL" = outputs results of "RRRC", "FRRC" and "RRFC" analyses - this is the default.
tempOrgCode,	default FALSE; if TRUE, then code from version 0.0.1 of RJafroc is used (see RJafroc_0.0.1.tar). This is intended to check against errors that crept in subsequent to the original version as I attempted to improve the organization of the code and the output. As implicit in the name of this temporary flag, it will eventually be removed.

Value

For method = "DBM" the returned list contains 4 dataframes:

FOMs	Contains foms, trtMeans and trtMeanDiffs: see return of UtilFigureOfMerit
ANOVA	Contains TRCAnova, VarCom, IndividualTrt and IndividualRdr ANOVA tables of pseudovalues
RRRC	Contains results of "RRRC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt
FRRC	Contains results of "FRRC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt, ciDiffTrtEachRdr
RRFC	Contains results of "RRFC" analyses: FTests, ciDiffTrt, ciAvgRdrEachTrt

For method = "OR" the return list contains 4 dataframes:

FOMs	Contains foms, trtMeans and trtMeanDiffs: UtilFigureOfMerit
ANOVA	Contains TRAnova, VarCom, IndividualTrt and IndividualRdr ANOVA tables of FOM values
RRRC	Contains results of "RRRC" analyses - same organization as DBM, see above
FRRC	Contains results of "FRRC" analyses - ditto
RRFC	Contains results of "RRFC" analyses- ditto

References

Dorfman DD, Berbaum KS, Metz CE (1992) ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method, Invest. Radiol. 27, 723-731.

Obuchowski NA, Rockette HE (1995) Hypothesis Testing of the Diagnostic Accuracy for Multiple Diagnostic Tests: An ANOVA Approach with Dependent Observations, Communications in Statistics: Simulation and Computation 24, 285-308.

Hillis SL (2014) A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data, Statistics in medicine 33, 330-360.

Chakraborty DP (2017) Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, CRC Press, Boca Raton, FL. https://www.routledge.com/Observer-Performance-Methods-for-Diagnostic-Imaging-Foundations-Modeling/Chakraborty/p/book/9781482214840

Examples

StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "DBM") 
StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "OR")
## following is split-plot-c analysis using a simulated split-plot-c dataset
StSignificanceTesting(datasetFROCSpC, FOM = "wAFROC", method = "OR")


StSignificanceTesting(dataset05, FOM = "wAFROC")
StSignificanceTesting(dataset05, FOM = "HrAuc", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "SongA1", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "SongA2", method = "DBM") 
StSignificanceTesting(dataset05, FOM = "wAFROC1", method = "DBM")
StSignificanceTesting(dataset05, FOM = "AFROC1", method = "DBM")
StSignificanceTesting(dataset05, FOM = "AFROC", method = "DBM")
 

RJafroc documentation built on Nov. 10, 2022, 5:45 p.m.