simulate_ab_test: Simulate a Bayesian A/B Test

In convoyinc/abayes: Bayesian A/B Testing

Description

Given true data generating distributions and prior distributions for variants A and B, simulate the data, calculate the necessary statistics and declare one of the tests a winner.

Usage

simulate_ab_test(data_dists, priors, loss_threshold, obs_per_round = 1000,
  max_rounds = 100, sim_batch_size = 1e+05)

Arguments

data_dists	A named list of distribution objects. This list specifies the distributions that are used to generate data in simulations. Currently, the list must only have elements named 'a' and 'b'. See Details for more information.
priors	A named list of distribution objects. This list specifies the distributions that are used as priors when estimating some parameter from the data generating distribution. Currently, the list must only have elements named 'a' and 'b'. See Details for more information.
loss_threshold	A positive number that identifies a bound for the expected loss for each variant. Once the expected loss is below this bound, the experiment is concluded.
obs_per_round	A positive number that represents how many observations, across both variants, are generated before we update the prior distributions and evaluate the expected loss. This number must be divisible by the number of variants used. Default is 1000.
max_rounds	A positive integer that specifies the maximum number of times that we will evaluate the expected loss on both experiments. Default is 100.
sim_batch_size	A positive integer that specifies how much data is simulated when evaluating the expected loss for variants that do not have an analytic solution (i.e. normal data).

Details

In order to create data_dists and priors, you need to use the following for data generating distributions: bernoulli_dist, normal_dist, poisson_dist and the following for prior distributions beta_dist, normal_gamma_dist, beta_dist,

Value

A list that contains the name of the winning variant, the number of observations used, the loss of the decision, whether the test finished, the metrics from each round, and the raw data.

See Also

bernoulli_dist normal_dist poisson_dist beta_dist normal_gamma_dist beta_dist

convoyinc/abayes documentation built on May 12, 2019, 1:34 a.m.