designSaviZ: Design a Safe Anytime-Valid Experiment to Test Means with a Z...
In safestats: Safe Anytime-Valid Inference

designSafeZ

R Documentation

Design a Safe Anytime-Valid Experiment to Test Means with a Z Test

Description

A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant mean difference meanDiffMin.

Scenario 1.a: Goal: "nPlan" and optimal E-variable. Given: meanDiffMin and power.
Scenario 1.b: Goal: an optimal E-variable. Given: meanDiffMin only.
Scenario 2: Goal: "power" and optimal E-variable. Given: meanDiffMin and nPlan.
Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable. Given: power and nPlan.
Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.

Usage

designSaviZ(
  meanDiffMin = NULL,
  power = NULL,
  nPlan = NULL,
  alpha = 0.05,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  meanDiffTrue = NULL,
  beta = NULL,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  lowEsTrue = 0.01,
  highEsTrue = 3,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  betaDefault = 0.2,
  highN = 10000L,
  wantSampling = TRUE,
  ...
)

Arguments

`meanDiffMin`	numeric that defines the minimal relevant mean difference, the smallest population mean difference that we would like to detect (with sufficient power).
`beta`	numerical in (0,1). Old parameter now replaced by the power parameter
`nPlan`	optional numeric vector of length at most 2, see scenario 2 and 3 above.
`alpha`	numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha.
`h0`	numeric, representing the null value, default h0=0.
`alternative`	a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less".
`sigma`	numeric > 0 representing the assumed population standard deviation used to scale the data.
`kappa`	the true population standard deviation. Default kappa=sigma.
`testType`	either one of "oneSample", "paired", "twoSample".
`ratio`	numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1.
`parameter`	numeric, an optional savi test defining parameter. Default set to `NULL`. and adapts to meanDiffMin and eType, see `matchEParameterWith` for details.
`eType`	character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior.
`wantSamplePaths`	logical, if `TRUE` then also outputs the sample paths.
`lowEsTrue`	numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a.
`highEsTrue`	numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a.
`pb`	logical, if `TRUE`, then show progress bar.
`seed`	integer, seed number.
`nSim`	integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi z test under continuous monitoring.
`nBoot`	integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi z test under continuous monitoring,or for the computation of the logarithm of the implied target.
`relevanceTest`	logical, if `TRUE` then impose a rule to stop for minimal efficiency if e <= alphaRelevance. Default `FALSE`.
`...`	further arguments to be passed to or from methods.
`power`	numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore.
`meanDiffTrue`	numeric, data governing mean difference used for simulations. Default meanDiffTrue=meanDiffMin.
`wantSimData`	logical, if `TRUE` then also output the simulated data
`relevanceSize`	numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin)
`alphaRelevance`	numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power.
`betaDefault`	numeric, defaulting value for 1-power and alphaRelevance
`highN`	integer, largest possible sampling horizon. This might be the largest n that we are able to fund, which by default is set to 1e4L. Typically, highN is not used, as the function `computeNPlanBatchSaviZ()` tries to find the sampling horizon. If all fails, then use highN as the sampling horizon.
`wantSampling`	logical, default TRUE so sampling paths are drawn. For instance, if meanDiffMin and power, are given, then nPlan (scenario 1a) is derived by sampling. Set this to FALSE, whenever we want to run a minimal efficacy test without needing to know nPlan

Details

Every scenario returns an E-variable adapted to the input. Scenario 1.a, for instance, outputs the parameter of the provided eType (default mom) savi test, see matchEParameterWith for details, and nPlan. The nPlan is based on samples paths drawn under meanDiffTrue (if not specified, then meanDiffTrue=meanDiffMin by default). The resulting nPlan corresponds to the power (say 80%) quantile of the first-passage time distribution associated with E crossing threshold 1/alpha.

Value

Returns a saviDesign object that includes:

parameter: the savi test defining parameter, see matchEParameterWith.
alpha: the tolerable type I error provided by the user.
pilot: logical, specifying whether it's a pilot design, which occurs when saviZTest is called without a designObj.
testName: "Z-Test".
call: the expression with which this function is called.

Functions

designSafeZ(): Deprecated version of designSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011. Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

# Scenario 1.b: Goal: an E-variable
designObj <- designSaviZ(meanDiffMin=0.8)

# Scenario 1.a: Goal: "nPlan" and optimal E-variable.
designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100)

plot(designObj)

# Scenario 1a. with relevance testing, also stopping for practically null
designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100,
                         relevanceTest=TRUE)

plot(designObj)

# Scenario 2: Goal: "power" and optimal E-variable
designObj <- designSaviZ(meanDiffMin=0.8, nPlan=16, nSim=100)

# Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable
designObj <- designSaviZ(power=0.7, nPlan=16)

# Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.
designObj <- designSaviZ(nPlan=16)

safestats documentation built on Sept. 6, 2026, 1:06 a.m.