designSaviT: Design a Safe Anytime-Valid Experiment to Test Means with a Z...
In safestats: Safe Anytime-Valid Inference

designSafeT

R Documentation

Design a Safe Anytime-Valid Experiment to Test Means with a Z Test

Description

A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant standarised mean difference deltaMin.

Scenario 1.a: Goal: "nPlan" and optimal E-variable. Given: deltaMin and power.
Scenario 1.b: Goal: an optimal E-variable. Given: deltaMin only.
Scenario 2: Goal: "power" and optimal E-variable. Given: deltaMin and nPlan.
Scenario 3.a: Goal: "deltaMin" and optimal E-variable. Given: power and nPlan.
Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.

Usage

designSaviT(
  deltaMin = NULL,
  power = NULL,
  nPlan = NULL,
  alpha = 0.05,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  deltaTrue = NULL,
  sigma = 1,
  sigma2 = sigma,
  lowEsTrue = 0.01,
  highEsTrue = 3,
  varEqual = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  betaDefault = 0.2,
  highN = 10000L,
  wantSampling = TRUE,
  nuMin = 2,
  ...
)

Arguments

`deltaMin`	numeric that defines the minimal relevant standardised mean difference, the smallest population effect size that we would like to detect (with sufficient power).
`beta`	numerical in (0,1). Old parameter now replaced by the power parameter
`nPlan`	optional numeric vector of length at most 2, see scenario 2 and 3 above.
`alpha`	numeric in (0, 1) that specifies the tolerable type I error and the null rejection rule e >= 1/alpha.
`h0`	numeric, representing the null value, default h0=0.
`alternative`	a character string specifying the alternative hypothesis. Must be one of "twoSided" (default), "greater" or "less".
`testType`	either one of "oneSample", "paired", "twoSample".
`ratio`	numeric > 0 representing the randomisation ratio of condition 2 over condition 1. If testType is not equal to "twoSample", or if nPlan is of length(1) then ratio=1.
`parameter`	numeric, an optional savi test defining parameter. Default set to `NULL`. and adapts to meanDiffMin and eType, see `matchEParameterWith` for details.
`eType`	character one of "mom", "grow", "eGauss", and "eCauchy". "mom" is default and uses a non-local moment prior with bump(s) at meanDiffMin, "grow" uses point prior(s) at meanDiffMin, "eGauss" a zero-centred normal prior, "eCauchy" a zero centred Cauchy prior.
`wantSamplePaths`	logical, if `TRUE` then also outputs the sample paths.
`lowEsTrue`	numeric, lower bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a.
`highEsTrue`	numeric, upper bound for the candidate set of the targeted minimal clinically relevant effect size for scenario 3.a.
`pb`	logical, if `TRUE`, then show progress bar.
`seed`	integer, seed number.
`nSim`	integer > 0, the number of simulations needed to compute power or the number of samples paths for the savi t test under continuous monitoring.
`nBoot`	integer > 0 representing the number of bootstrap samples to assess the accuracy of the approximations of the power, the number of samples for the savi t test under continuous monitoring,or for the computation of the logarithm of the implied target.
`...`	further arguments to be passed to or from methods, but mainly to perform do.calls.
`deltaTrue`	numeric, data governing effect size used for simulations. Default deltaTrue=deltaMin.
`power`	numeric in (0, 1) that specifies the desired power, that is, the targetted chance to stop in favour of the alternative over the null hypothesis, when the alternative holds true. Note that prior to version 0.8.8 power <- 1-beta. The "beta" argument does not need to be specified anymore.
`wantSimData`	logical, if `TRUE` then also output the simulated data
`sigma`	numeric > 0 representing the population standard deviation used for the test.
`sigma2`	numeric > 0 representing the population standard deviation used for the test, for the second group in a two-sample t-test
`varEqual`	a logical variable indicating whether to treat the two variances as being equal. Default varEqual=TRUE.
`relevanceTest`	logical, if `TRUE` then impose a rule to stop for minimal efficiency if e <= alphaRelevance. Default `FALSE`.
`relevanceSize`	numeric, the minimal clinical relevant mean difference that we do not want to miss under the alternative. Default relevanceSize=NULL implies relevanceSize=abs(meanDiffMin)
`alphaRelevance`	numeric, the threshold for relevance test. Taken to be minimum of alpha and 1-power.
`betaDefault`	numeric, defaulting value for 1-power and alphaRelevance
`highN`	integer, largest possible sampling horizon. This might be the largest n that we are able to fund, which by default is set to 1e4L. Typically, highN is not used, as the function `computeNPlanBatchSaviT()` tries to find the sampling horizon. If all fails, then use highN as the sampling horizon.
`wantSampling`	logical, default TRUE so sampling paths are drawn. For instance, if meanDiffMin and power, are given, then nPlan (scenario 1a) is derived by sampling. Set this to FALSE, whenever we want to run a minimal efficacy test without needing to know nPlan
`nuMin`	numeric > 0, the minimum degrees of freedom under which the results are trivial, thus, 1.

Details

Every scenario returns an E-variable adapted to the input. Scenario 1.a, for instance, outputs the parameter of the provided eType (default mom) savi test, see matchEParameterWith for details, and nPlan. The nPlan is based on samples paths drawn under deltaTrue (if not specified, then deltaTrue=deltaMin by default). The resulting nPlan corresponds to the power (say 80%) quantile of the first-passage time distribution associated with E crossing threshold 1/alpha.

Value

Returns an object of class 'saviDesign'. An object of class 'saviDesign' is a list containing at least the following components:

parameter: the savi test defining parameter, see matchEParameterWith.
alpha: the tolerable type I error provided by the user.
pilot: logical, specifying whether it's a pilot design, which occurs when saviTTest is called without a designObj.
testName: "T-Test".
call: the expression with which this function is called.

Functions

designSafeT(): Deprecated version of designSaviT
designSafeT2WantBeta(): Deprecated version of designSaviT2WantBeta
designSafeT3WantEsMin(): Deprecated version of designSafeT3WantEsMin
designSafeT3bWantParameter(): Deprecated version of designSaviT3bWantParameter
computeNPlanBatchSafeT(): Deprecated version of computeNPlanBatchSaviT
computeMinEsBatchSafeT(): Deprecated version of computeMinEsBatchSaviT

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011. Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

# Scenario 1.b: Goal: an E-variable
designObj <- designSaviT(deltaMin=0.8)

# Scenario 1.a: Goal: "nPlan" and optimal E-variable.
designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100)

plot(designObj)

# Scenario 1a. with relevance testing, also stopping for practically null
designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100,
                         relevanceTest=TRUE)

plot(designObj)

# Scenario 2: Goal: "power" and optimal E-variable
designObj <- designSaviT(deltaMin=0.8, nPlan=16, nSim=100)

# Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable
designObj <- designSaviT(power=0.7, nPlan=16)

# Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.
designObj <- designSaviT(nPlan=16)

safestats documentation built on Sept. 6, 2026, 1:06 a.m.