evaluateBiomarkerTestSets: Wrapper function to evaluate all test sets or a specified...

In RIbench: Benchmark Suite for Indirect Methods for RI Estimation

Wrapper function to evaluate all test sets or a specified subset for a specified algorithm.

Description

Wrapper function to evaluate all test sets or a specified subset for a specified algorithm.

Usage

evaluateBiomarkerTestSets(
  workingDir = "",
  algoName = "refineR",
  algoFunction = "findRI",
  libs = "refineR",
  sourceFiles = NULL,
  params = NULL,
  requireDecimals = FALSE,
  requirePercentiles = FALSE,
  subset = "all",
  timeLimit = 14400,
  verbose = TRUE,
  showWarnings = FALSE,
  ...
)

Arguments

workingDir	(character) specifying the working directory: Results will be stored in 'workingDir/Results/algo/biomarker' and data will be used from 'workingDir/Data/biomarker'
algoName	(character) specifying the name of the algorithm that is evaluated
algoFunction	(character) specifying the name of the function needed for estimating RIs
libs	(list) containing all libraries needed for executing the algorithm
sourceFiles	(list) containing all source files needed for executing the algorithm
params	(list) with additional parameters needed for calling algoFunction
requireDecimals	(logical) indicating whether the algorithm needs the number of decimal places (TRUE) or not (FALSE, default)
requirePercentiles	(logical) indicating whether only percentiles and no model is estimated
subset	(character, numeric, or data.frame) to specify for which subset the algorithm should be executed. character options: 'all' (default) for all test sets; a distribution type: 'normal', 'skewed', 'heavilySkewed', 'shifted'; a biomarker: 'Hb', 'Ca', 'FT4', 'AST', 'LACT', 'GGT', 'TSH', 'IgE', 'CRP', 'LDH'; 'Runtime' for runtime analysis subset; numeric option: number of test sets per biomarker, e.g. 10; data.frame: customized subset of table with test set specifications
timeLimit	(integer) specifying the maximum amount of time in seconds allowed to execute one single estimation (default: 14400 sec (4h))
verbose	(logical) indictaing if the progress counter should be shown (default: TRUE)
showWarnings	(logical) indicating whether warnings from the call to the indirect method/algorithm should be shown (default: FALSE)
...	additional arguments to be passed to the method, e.g. specified in- and output directory ('inputDir', 'outputDir')

Value

(data frame) containing information about the test sets where the algorithm terminated the R session or failed to report a result

Author(s)

Tatjana Ammer [e-mail]

Examples

## Not run: 
# The evaluation of all test sets can take several hours depending on 
#   the computation time of the algorithm.
# Wrapper function for indirect method required, see vignette("RIbench_package")
# Ensure that 'generateBiomarkerTestSets()' is called with the same workingDir 
#    before calling this function. 

# first generic example
evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModel', libs = c('myOwnAlgo'), 
		sourceFiles = list("C:\\Temp\\MyAlgoWrapper.R"), 
	requireDecimals = FALSE, requirePercentiles = FALSE,
	subset ='all', timeLimit = 14400)


# second example, evaluation for only 'Calcium' test sets.
progress <- evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
			algoFunction = 'estimateModel', libs = c('myOwnAlgo'), subset = "Ca")


# third example, evaluation for only a subset testsets that follow a skewed distribution.
progress <- evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModel', libs = c('myOwnAlgo'), subset = "skewed") 


# forth example, evaluation for a subset of 3 testsets per biomarker. 
progress <- evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModel', libs = c('myOwnAlgo'), subset = 3)


# fifth example, evaluation for a customized subset with all test sets that have 
# 	a pathological fraction <= 30%. 
testsets <- loadTestsetDefinition()
progress <- evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModel', libs = c('myOwnAlgo'), 
		subset = testsets[testsets$fractionPathol <= 0.3,] )


# sixth example, evaluation forwarding additional parameters to the 'algoFunction'
progress <- evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModel', libs = c('myOwnAlgo'), 
		sourceFiles = list("Test_RIEst_2pBoxCox"), params = list("model='2pBoxCox'"))


# seventh example, evaluation for indirect method that requires the number of 
#	decimal points as input 
evaluateBiomarkerTestSets(workingDir = tempdir(), algoName = 'myOwnAlgo', 
		algoFunction = 'estimateModelDec', libs = c('myOwnAlgo'), 
		sourceFiles = "C:\\Temp\\Test_RIEst_dec.R", requireDecimals = TRUE)


# eigth example, evaluation for indirect method that directly estimates the percentiles
evaluateBiomarkerTestSets(workingDir = tempdir(), algoName="myOwnAlgo", 
		algoFunction="estimateRIs", libs="myOwnAlgo", 
		sourceFiles = "C:\\Temp\\Test_RIEst.R", requirePercentiles=TRUE) 

## End(Not run)

RIbench documentation built on Nov. 28, 2022, 1:11 a.m.