B Testing

Run multiple simulations of an A/B test in order to evaluate various properties

1
2
3

investigate_simulations(num_sims, priors, loss_threshold, data_dists = NULL,
  sampling_distribution = NULL, obs_per_round = 1000, max_rounds = 100,
  sim_batch_size = 1e+05, num_cores = NULL)

`num_sims`	A positive integer that specifies how many simulations to perform.
`priors`	A named list of distribution objects. This list specifies the distributions that are used as priors when estimating some parameter from the data generating distribution. Currently, the list must only have elements named `'a'` and `'b'`. See Details for more information.
`loss_threshold`	A positive number that identifies a bound for the expected loss for each variant. Once the expected loss is below this bound, the experiment is concluded.
`data_dists`	A named list of distribution objects. This list specifies the distributions that are used to generate data in simulations. Currently, the list must only have elements named `'a'` and `'b'`. See Details for more information.
`sampling_distribution`	An list of distribution objects that specifies how the data generating distributions should be created.
`obs_per_round`	A positive number that represents how many observations, across both variants, are generated before we update the prior distributions and evaluate the expected loss. This number must be divisible by the number of variants used. Default is `1000`.
`max_rounds`	A positive integer that specifies the maximum number of times that we will evaluate the expected loss on both experiments. Default is `100`.
`sim_batch_size`	A positive integer that specifies how much data is simulated when evaluating the expected loss for variants that do not have an analytic solution (i.e. normal data).
`num_cores`	How many cores to use in the parallelization of tests. Default is `NULL`, which means no parallelization.