designSafeTwoProportions: Designs a Safe Experiment to Test Two Proportions in Stream...

In safestats: Safe Anytime-Valid Inference

Designs a Safe Experiment to Test Two Proportions in Stream Data

Description

The design requires the number of observations one expects to collect in each group in each data block. I.e., when one expects balanced data, one could choose na = nb = 1 and would be allowed to analyse the data stream each time a new observation in both groups has come in. The best results in terms of power are achieved when the data blocks are chosen as small as possible, as this allows for analysing and updating the safe test as often as possible, to fit the data best. Further, the design requires two out of the following three parameters to be known:

• the power one aims to achieve (1 - beta),

• the minimal relevant difference between the groups (delta)

• the number of blocks planned (nBlocksPlan),

where the unknown out of the three will be estimated. In the case of an exploratory "pilot" analysis, one can also only provide the number of blocks planned.

Usage

designSafeTwoProportions(
  na,
  nb,
  nBlocksPlan = NULL,
  beta = NULL,
  delta = NULL,
  alternativeRestriction = c("none", "difference", "logOddsRatio"),
  alpha = 0.05,
  pilot = "FALSE",
  hyperParameterValues = NULL,
  previousSafeTestResult = NULL,
  M = 1000,
  simThetaAMin = NULL,
  simThetaAMax = NULL
)

Arguments

na	number of observations in group a per data block
nb	number of observations in group b per data block
nBlocksPlan	planned number of data blocks collected
beta	numeric in (0, 1) that specifies the tolerable type II error control necessary to calculate both "nBlocksPlan" and "delta". Note that 1-beta defines the power.
delta	a priori minimal relevant divergence between group means b and a, either a numeric between -1 and 1 for no alternative restriction or a restriction on difference, or a real for a restriction on the log odds ratio.
alternativeRestriction	a character string specifying an optional restriction on the alternative hypothesis; must be one of "none" (default), "difference" (difference group mean b minus group b) or "logOddsRatio" (the log odds ratio between group means b and a).
alpha	numeric in (0, 1) that specifies the tolerable type I error control –independent on n– that the designed test has to adhere to. Note that it also defines the rejection rule e10 > 1/alpha.
pilot	logical, specifying whether it's a pilot design.
hyperParameterValues	named list containing numeric values for hyperparameters betaA1, betaA2, betaB1 and betaB2, with betaA1 and betaB1 specifying the parameter equivalent to shape1 in stats::dbeta for groups A and B, respectively, and betaA2 and betaB2 equivalent to shape2. By default chosen to optimize evidence collected over subsequent experiments (REGRET). Pass in the following format: list(betaA1 = numeric1, betaA2 = numeric2, betaB1 = numeric3, betaB2 = numeric4).
previousSafeTestResult	optionally, a previous safe test result can be provided. The posterior of the hyperparameters of this test is then used for the hyperparameter settings. Default NULL.
M	number of simulations used to estimate power or nBlocksPlan. Default 1000.
simThetaAMin	minimal event rate in control group to simulate nPlan or power for. Can be specified when specifically interested in planning studies for specific event rates. Default NULL, then the entire parameter space (possibly restricted by delta) is used for simulation.
simThetaAMax	maximal event rate in control group to simulate nPlan or power for. Default NULL.

Value

Returns a 'safeDesign' object that includes:

nPlan

the sample size(s) to plan for. Computed based on beta and meanDiffMin, or provided by the user if known.

parameter

the safe test defining parameter: here the hyperparameters.

esMin

the minimally clinically relevant effect size provided by the user.

alpha

the tolerable type I error provided by the user.

beta

the tolerable type II error specified by the user.

alternative

any of "twoSided", "greater", "less" based on the alternativeRestriction provided by the user.

testType

here 2x2

pilot

logical, specifying whether it's a pilot design.

call

the expression with which this function is called.

Examples

#plan for an experiment to detect minimal difference of 0.6 with a balanced design
set.seed(3152021)
designSafeTwoProportions(na = 1,
                         nb = 1,
                         alpha = 0.1,
                         beta = 0.20,
                         delta = 0.6,
                         alternativeRestriction = "none",
                         M = 75)

#safe analysis of a pilot: number of samples already known
designSafeTwoProportions(na = 1,
                          nb = 1,
                          nBlocksPlan = 20,
                          pilot = TRUE)

#specify own hyperparameters
hyperParameterValues <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
designSafeTwoProportions(na = 1,
                         nb = 1,
                         alpha = 0.1,
                         beta = 0.20,
                         delta = 0.6,
                         hyperParameterValues = hyperParameterValues,
                         alternativeRestriction = "none",
                         M = 75)

#restrict range of proportions for estimating nPlan in the control group
designSafeTwoProportions(na = 1,
                         nb = 1,
                         beta = 0.20,
                         delta = 0.3,
                         alternativeRestriction = "none",
                         M = 75,
                         simThetaAMin = 0.1, simThetaAMax = 0.2)

safestats documentation built on Nov. 24, 2022, 5:07 p.m.