simulateTwoProportions: Compare Different Hyperparameter Settings for Safe Tests of...

In safestats: Safe Anytime-Valid Inference

Compare Different Hyperparameter Settings for Safe Tests of Two Proportions.

Description

Simulates for a range of divergence parameter values (differences or log odds ratios) the worst-case stopping times (i.e., number of data blocks collected) and expected stopping times needed to achieve the desired power for each hyperparameter setting provided.

Usage

simulateTwoProportions(
  hyperparameterList,
  alternativeRestriction = c("none", "difference", "logOddsRatio"),
  deltaDesign = NULL,
  alpha,
  beta,
  na,
  nb,
  deltamax = 0.9,
  deltamin = 0.1,
  deltaGridSize = 8,
  M = 100,
  maxSimStoptime = 10000,
  thetaAgridSize = 8
)

Arguments

hyperparameterList	list object, its components hyperparameter lists with a format as described in designSafeTwoProportions().
alternativeRestriction	a character string specifying an optional restriction on the alternative hypothesis; must be one of "none" (default), "difference" (difference group mean b minus group b) or "logOddsRatio" (the log odds ratio between group means b and a).
deltaDesign	optional; when using a restricted alternative, the value of the divergence measure used. Either a numeric between -1 and 1 for a restriction on difference, or a real for a restriction on the log odds ratio.
alpha	numeric in (0, 1) that specifies the tolerable type I error control –independent on n– that the designed test has to adhere to. Note that it also defines the rejection rule e10 > 1/alpha.
beta	numeric in (0, 1) that specifies the tolerable type II error control in the study. Necessary to calculate the worst case stopping time.
na	number of observations in group a per data block
nb	number of observations in group b per data block
deltamax	maximal effect size to calculate power for; between -1 and 1 for designs without restriction or a restriction on difference; real number for a restriction on the log odds ratio. Default 0.9.
deltamin	minimal effect size to calculate power for; between -1 and 1 for designs without restriction or a restriction on difference; real number for a restriction on the log odds ratio. Default 0.1.
deltaGridSize	numeric, positive integer: size of grid of delta values worst case and expected sample sizes are simulated for.
M	number of simulations used to estimate sample sizes. Default 100.
maxSimStoptime	maximal stream length in simulations; when the e value does not reach the rejection threshold before the end of the stream, the maximal stream length is returned as the stopping time. Default 1e4.
thetaAgridSize	numeric, positive integer: size of the grid of probability distributions examined for each delta value to find the worst case sample size over.

Value

Returns an object of class "safe2x2Sim". An object of class "safe2x2Sim" is a list containing at least the following components:

simData

A data frame containing simulation results with worst case and expected stopping times for each hyperparameter setting, for the specified or default range of effect sizes.

alpha

the significance threshold used in the simulations

beta

the type-II error control used in the simulations

deltaDesign

the value of restriction on the alternative hypothesis parameter space used for the E variables in the simulations

restriction

the type of restriction used for the E variables in the simulation

hyperparameters

list of the hyperparameters tested in the simulation

Examples

priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18)
priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1)

simResult <- simulateTwoProportions(
  hyperparameterList = list(priorList1, priorList2, priorList3),
  alternativeRestriction = "none",
  alpha = 0.1, beta = 0.2, na = 1, nb = 1,
  deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3,
  M = 10
  )

print(simResult)
plot(simResult)

safestats documentation built on Nov. 24, 2022, 5:07 p.m.