designSavi2x2: Design a Safe Anytime-Valid Experiment to Test Two...
In safestats: Safe Anytime-Valid Inference

designSavi2x2

R Documentation

Design a Safe Anytime-Valid Experiment to Test Two Proportions

Description

A designed experiment requires (1) a number of data blocks nBlocksPlan to plan for, and (2) a savi test defining parameter: eType and, with it, the effect measure. Both effects are A minus B and anchored on thetaB: propDiff = thetaA - thetaB and logOdds = logit(thetaA) - logit(thetaB), so "greater" means group A has the larger proportion, as x - y > 0 does in stats::t.test().

Usage

designSavi2x2(
  na,
  nb,
  nBlocksPlan = NULL,
  propDiffMin = NULL,
  logOddsMin = NULL,
  alpha = 0.05,
  power = NULL,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  eType = c("eBeta", "grow", "eGauss"),
  betaParameter = NULL,
  gaussParameter = NULL,
  runningIntersection = NULL,
  nSim = 1000L,
  nBoot = 1000L,
  nMax = 10000L,
  seed = NULL,
  wantSamplePaths = FALSE,
  pb = TRUE
)

Arguments

`na`	number of observations in group A per data block, one value or one per block.
`nb`	number of observations in group B per data block, one value or one per block.
`nBlocksPlan`	planned number of data blocks collected, see scenario 2 and 3 above.
`propDiffMin`	numeric in (-1, 1) that defines the minimal relevant difference in proportions `thetaA - thetaB`, the smallest difference that we would like to detect (with sufficient power).
`logOddsMin`	numeric that defines the minimal relevant log odds ratio `logit(thetaA) - logit(thetaB)`.
`alpha`	numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha.
`power`	numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true.
`h0`	numeric, representing the null value, default h0=0. Only `h0 = 0` is currently supported.
`alternative`	a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less", where "greater" means that group A has the larger proportion.
`eType`	character one of "eBeta", "grow", and "eGauss". "eBeta" is default and uses a Beta prior on each proportion, "grow" uses a point prior at the minimal effect, "eGauss" a normal prior on the log odds ratio.
`betaParameter`	`list(betaA1, betaA2, betaB1, betaB2)`, the Beta prior shapes on `thetaA` and `thetaB` for "eBeta", each a single finite positive number; `NULL` means `1 / (2 * na)` and `1 / (2 * nb)`, taking block 1's sizes with a warning when they vary by block. A given prior keeps its own block 1: the test does not replace it by the UMP e-value. For "grow" on propDiff, the test puts `Beta(betaA1, betaA2)` on `thetaA` rescaled to its feasible interval on the curve, B's shapes unused, and `NULL` means uniform; planning ignores it.
`gaussParameter`	`list(mean, sd)`, the Normal prior on `logOdds` for "eGauss", restricted to the grid `⁠(-20, 20)⁠` and to the side of a one-sided `alternative`; `NULL` means `list(mean = 0, sd = 1)`.
`runningIntersection`	logical, if `TRUE` then intersect each row of the blockwise confidence sequence with the previous one; `NULL` keeps the constructor's `FALSE`.
`nSim`	integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi 2x2 test under continuous monitoring.
`nBoot`	integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, or the number of blocks for the savi 2x2 test under continuous monitoring.
`nMax`	integer > 0, maximum number of data blocks in each sample path.
`seed`	integer, seed number. Default seed=NULL yields seed=2026.
`wantSamplePaths`	logical, if `TRUE` then also outputs the sample paths.
`pb`	logical, if `TRUE`, then show progress bar.

Details

"eBeta" (propDiff) and "eGauss" (logOdds) are unrestricted: a minimal effect is an error, a power is dropped with a warning, and nBlocksPlan is kept as the planned block count. Each reads one prior, betaParameter or gaussParameter; NULL keeps the default, which savi2x2TestStat() fills in. A one-sided alternative acts in the UMP first block, and for "eGauss" also restricts the prior to that side.
"grow" plugs in exactly one of propDiffMin, logOddsMin as the fixed alternative and reads no prior. As in designSaviZ(), a value whose sign contradicts a one-sided alternative is flipped with a warning; "twoSided" uses the magnitude. Zero is an error.

For "grow" on propDiff the design involves alpha and the three quantities: (1) nBlocksPlan, (2) power, and (3) a minimal relevant difference in proportions propDiffMin, simulated at the worst-case baseline of sampleStoppingTimesSavi2x2().

Scenario 1a: Goal: "nBlocksPlan" and optimal E-variable. Given: propDiffMin and power.
Scenario 2: Goal: "power" and optimal E-variable. Given: propDiffMin and nBlocksPlan.
Scenario 3: Goal: "propDiffMin" and optimal E-variable. Given: power and nBlocksPlan.

A minimal effect alone designs without simulation. Any other combination is an error, except that logOddsMin drops a power with a warning. Per-block size vectors supply nBlocksPlan when it is NULL. Other arguments are taken as given; only alpha and power in ⁠(0, 1)⁠, equal lengths of na and nb, at most one minimal effect, and the minimal effects' ranges are checked.

Value

Returns a saviDesign object that includes:

nPlan: the planned sample size(s): list(na, nb), with nBlocksPlan when given or planned.
parameter: for "eBeta" and "eGauss", the prior as one named string, or "default" when NULL; absent for "grow", whose minimal effect is esMin.
esMin: the minimal relevant effect size provided by the user, or found in scenario 3, signed and named propDiff or logOdds; NULL for "eBeta" and "eGauss".
alpha: the tolerable type I error provided by the user.
power: the desired power provided by the user, or the worst-case power found in scenario 2.
alternative: any of "twoSided", "greater", "less" provided by the user.
eType: any of "eBeta", "grow", "eGauss" provided by the user.
h0: the null value, 0.
betaParameter: for "eBeta" and "grow" on propDiff, the Beta prior given, NULL for the default.
gaussParameter: for "eGauss", the Normal prior given, NULL for the default.
runningIntersection: logical, as provided by the user.
designScenario: "1a", "2" or "3" when planned.
nPlanTwoSe, bootObjNBlocksPlan, nMean, nMeanTwoSe, bootObjNMean: scenario 1a: the bootstrap summaries of the planned block count and of the mean stopping time.
powerTwoSe, bootObjPower: scenario 2: the bootstrap summaries of the power.
worstCaseThetaA, worstCaseThetaB, breakVector, samplePaths: scenarios 1a and 2: the worst baseline and its simulated paths.
testType: here 2x2
testName: "Two Proportions".
call: the expression with which this function is called.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Turner, R., Ly, A., & Grünwald, P. D. (2024). Generic e-variables for exact sequential k-sample tests that allow for optional stopping. Journal of Statistical Planning and inference 230, 106116, https://doi.org/10.1016/j.jspi.2023.106116.

Examples

# Unrestricted test on propDiff with the default Beta prior
designSavi2x2(na = 10, nb = 10, eType = "eBeta")

# Grow on a minimal difference, no planning
designSavi2x2(na = 10, nb = 10, propDiffMin = 0.2, eType = "grow")

# Scenario 2: worst-case power at a planned block count
designSavi2x2(na = 10, nb = 10, propDiffMin = 0.3, nBlocksPlan = 8,
              eType = "grow", nSim = 50, pb = FALSE)

safestats documentation built on Oct. 5, 2026, 9:07 a.m.