sma_evaluate_files: Calculates measures to evaluate the performance of the model

In audrey-b/simanalyse: Facilitate Model Comparisons and Simulation Studies

Calculates measures to evaluate the performance of the model

Description

Calculates measures to evaluate the performance of the model The results are compared with the true parameter values by calculating performance measures such as bias, mean square error and coverage probability. R code can be used to customize the performance measures

Usage

sma_evaluate_files(
  measures = c("bias", "mse", "cpQuantile"),
  estimator = mean,
  alpha = 0.05,
  h_null = 0,
  parameters = NULL,
  monitor = ".*",
  deviance = FALSE,
  path = ".",
  folder = "analysis0000001",
  custom_funs = list(),
  custom_expr_before = "",
  custom_expr_after = "",
  progress = FALSE,
  options = furrr::furrr_options()
)

Arguments

measures	A vector of strings indicating which performance measures to calculate. Strings may include "bias", "E" (expectation), "cpQuantile" (coverage probability of quantile-based CrIs of level alpha), "LQuantile" (length of quantile-based CrIs of level alpha), "Epvar" (expected posterior variance), "Epsd" (expected posterior standard deviation), "rb" (relative bias), "br" (bias ratio), "var" (variance), "se" (standard error), "mse" (root mean square error), "rmse" (root mean square error), "rrmse" (relative root mean square error), "cv" (coefficient of variation), "power" (coverage probability of the value h_null of quantile-based CrIs of level alpha), "all" (all the measures)
estimator	A function, typically mean or median, for the Bayes estimator to use to calculate the performance measures.
alpha	Scalar representing the alpha level used to construct credible intervals. Default is 0.05.
h_null	Scalar representing the value used to evaluate power. Default is 0.
parameters	An nlist object (or list that can be coerced to nlist). True values of parameters to be used to calculate the performance measures.
monitor	A character vector (or regular expression if a string) specifying the names of the stochastic nodes in code to include in the summary. By default all stochastic nodes are included.
deviance	Whether to calculate measures for deviance.
path	A string. The object is read using this path. If a "derive" folder exists, the object is read from that folder, otherwise it is read from the "results" folder.
folder	A string for the name of the folder that contains the analysis.
custom_funs	A named list of functions to calculate over the mcmc samples. E.g. list(posteriormedian = median).
custom_expr_before	A string of R code to derive custom measures. This code is used BEFORE averaging over all simulations. E.g. "mse = (posteriormedian - parameters)^2". Functions from custom_funs may be used as well as the keywords 'parameters' (the true values of the parameters) and 'estimator' (the estimator defined in estimator).
custom_expr_after	A string of R code to derive additional custom measures. This code is used AFTER averaging over all simulations. E.g. "rmse = sqrt(mse)". Measures calculated from custom_expr_before may be used as well as the keyword 'parameters' (the true values of the parameters).
progress	A flag specifying whether to print a progress bar.
options	The future specific options to use with the workers.

Value

A flag.

audrey-b/simanalyse documentation built on May 20, 2022, 7:45 p.m.