compute_iters: Determine Number of Iterations to Simulate a Stochastic Model
In JohnNay/sa: Sensitivity Analysis for Complex Computational Models

Description Usage Arguments Details Value References Examples

compute_iters estimates a sufficient number of iterations for subsequent analysis of a simulation model.

compute_iters(abm, input_values, out, sample_count = 20, model_data = NULL,
  repeats = 30, thresh = 0.05, initial_iters = 5, max_iters = 1000,
  constraints = "none", parallel = FALSE, cores = NULL, verbose = FALSE,
  measure = c("coef_var", "var", "sd"))

`abm`	A function that takes each of the `input_values` as arguments. Specifically, a function that has at least three arguments: `parameters, out, iterations`.
`input_values`	List
`out`	Character vector length one to be passed an argument to the `abm` function to specify what outcome measure to use.
`sample_count`	Optional numeric vector length one specifying the number of samples for a given `iters` value that is being tested. If `repeats` is very high then this does not need to be as high.
`model_data`	Optional data.frame with the data that was used to build the model. This is used if one wants to ensure that all parameters tested are in the convex hull of the data used to build the model that is now being analyzed. This uses the WhatIf package in the Suggest field of the eat description file.
`repeats`	Optional numeric vector length one specifying the number of times to repeat the main loop of this algo. If `sample_count` is very high then this does not need to be as high.
`thresh`	Optional numeric vector length one. This is a hyper-parameter of this algorithm and may need to be experimented with by the user.
`initial_iters`	Optional numeric vector length one. Although optional, this is very problem dependent and so should usually be set with knowledge of the simulation model.
`max_iters`	Optional numeric vector length one. Although optional, this is very problem dependent and so should usually be set with knowledge of the simulation model.
`constraints`	Optional Character vector that is either "none" or is using only variable names that are specified in the input_values List argument. This character vector is evaluated in an environment created for the sampled data on the variables, and its evaluation results in a Logical vector that that subsets sampled.
`parallel`	Optional logical vector length one. Default is FALSE.
`cores`	Optional Numeric vector length one. Default is parallel::detectCores().
`verbose`	Optional logical vector. Default for this algorithm is FALSE because messages are created during an inner loop, i.e. very often.
`measure`	Optional character vector. One of `c("coef_var", "var", "sd")`. Don't use `coef_var` if the outcome values of the simulation model can take on negative values or are on an arbitrary scale; otherwise, use `coef_var`.

This is a function of the eat package. It takes an abm in function form and a list of input values. It returns a list with the result and diagnostic information.

We often deal with stochastic simulations, which requires a model run (a model run may comprise an arbitrary number of discrete time steps specific to the model) to be repeated >1 times with the same global parameter settings in order to reduce the variance of outcomes within a specified global ABM parameter setting low enough to allow for comparison of outcomes across parameter settings, i.e. for learning anything useful about the input-output relationships that define the ABM. The size of this experimental noise should be analyzed prior to executing simulation experiments and optimization routines. Lorscheid, I., Heine, B.O., & Meyer, M. (2012) suggest using the coefficient of variation, c_v = s/mu, where s is the standard deviation and mu is the mean. Because the coefficient of variation is a dimensionless and normalized measure of variance it can be used to investigate the variance of multiple simulation outcome variables.

List with the result and diagnostic information. List has elements: call; result; timing; and session. Timing is in seconds.

Lorscheid, I., Heine, B.O., & Meyer, M. (2012). Opening the "black box" of simulations: increased transparency and effective communication through the systematic design of experiments. Computational and Mathematical Organization Theory, 18 (1), 22-62.

Hendricks W, Robey K (1936) The sampling distribution of the coefficient of variation. Ann Math Stat 7:129-132

fake <- function(inputs, out, iterations) 
mean(rnorm(iterations, inputs[1], inputs[2]))
inputs <- lapply(list(.mean = NA, .sd = NA),
function(x) list(random_function = "qunif",
ARGS = list(min = 0.1, max = 0.1)))
hist(rnorm(15, 0.1, 0.1))
res <- compute_iters(fake, inputs, "hello", repeats = 20,
thresh = 1e-100, max_iters = 1e1000,
initial_iters = 100)
plot(res, ylab = "Coefficient of Variation")