monitorTrial: Group Sequential Monitoring of Simulated Efficacy Trials for...

In mjuraska/seqDesign: Simulation and Group-Sequential Monitoring of Randomized Treatment Efficacy Trials with Time-to-Event Endpoints

Group Sequential Monitoring of Simulated Efficacy Trials for the Event of Potential Harm, Non-Efficacy, and High Efficacy

Description

monitorTrial applies a group sequential monitoring procedure to data-sets generated by simTrial, which may result in modification or termination of each simulated trial.

Usage

monitorTrial(
  dataFile,
  stage1,
  stage2,
  harmMonitorRange,
  harmMonitorAlpha = 0.05,
  alphaPerTest = NULL,
  nonEffStartMethod = c("FKG", "fixed"),
  nonEffStartParams = NULL,
  nonEffIntervalUnit = c("counts", "time"),
  nonEffInterval,
  nonEffCohorts = list(times = NULL, timeUnit = "counts", timingCohort = list(lagTime =
    NULL, cohortInd = NULL), cohort1 = list(estimand = "cox", lagTime = NULL, cohortInd =
    NULL, nullVE = NULL, nominalAlphas = NULL)),
  effCohort = list(times = NULL, timeUnit = "counts", timingCohort = list(lagTime =
    NULL, cohortInd = NULL), nullVE = NULL, estimand = "cox", lagTime = NULL, cohortInd =
    NULL, nominalAlphas = NULL),
  stage1Eff = list(cohort = list(lagTime = 0, cohortInd = NULL), nullVE = NULL,
    nominalAlpha = NULL, estimand = "cox"),
  lowerVEnoneff = NULL,
  highVE,
  stage1VE,
  lowerVEuncPower = NULL,
  alphaHigh,
  alphaStage1,
  alphaUncPower = NULL,
  saveFile = NULL,
  saveDir = NULL,
  verbose = TRUE
)

Arguments

dataFile	if \code{saveDir} = \code{NULL}, a list returned by simTrial; otherwise a name (character string) of an .RData file created by simTrial
stage1	the final week of stage 1 in a two-stage fixed-follow-up trial
stage2	the final week of stage 2 in a two-stage fixed-follow-up trial, i.e., the maximum total follow-up time
harmMonitorRange	a 2-component numeric vector specifying the range of the pooled number of events (pooled over the placebo and vaccine arm accruing events the fastest) over which the type I error rate, specified in harmMonitorAlpha, shall be spent (per vaccine arm). Note that harmMonitorRange does not specify a range over which potential-harm stopping boundaries will be computed; instead, it specifies when potential-harm monitoring will start, and the range over which the type I error rate harmMonitorAlpha will be spent. If \code{nonEffStartMethod} = \code{"FKG"} or "fixed", then the second value is ignored and can be replaced with NA.
harmMonitorAlpha	a numeric value (0.05 by default) specifying the overall type I error rate for potential-harm monitoring (per vaccine arm). To turn off potential-harm monitoring, set harmMonitorAlpha equal to 0.00001.
alphaPerTest	a per-test nominal significance level for potential-harm monitoring. If NULL (default), a per-test significance level is calculated that yields the overall type I error rate of harmMonitorAlpha at the end of harmMonitorRange.
nonEffStartMethod	a character string specifying the method used for determining when non-efficacy monitoring is to start. The default method of Freidlin, Korn, and Gray (2010) ("FKG") calculates the minimal pooled event count (pooled over the placebo and vaccine arm accruing events the fastest) such that a hazard-ratio-based VE point estimate of 0% would result in declaring non-efficacy, i.e., the upper bound of the two-sided (1-\code{alphaNoneff}) 100\% confidence interval for VE based on the asymptotic variance of the log-rank statistic equals the non-efficacy threshold specified as component upperVEnonEff in the list nonEffStartParams. If this list component is left unspecified, the argument upperVEnonEff is used as the non-efficacy threshold. The alternative method ("fixed") starts non-efficacy monitoring at a fixed pooled event count (pooled over the placebo and vaccine arm accruing events the fastest) specified by component N1 in the list nonEffStartParams.
nonEffStartParams	a list with named components specifying parameters required by nonEffStartMethod (NULL by default)
nonEffIntervalUnit	a character string specifying whether intervals between two adjacent non-efficacy interim analyses should be event-driven (default option "counts") or calendar time-driven (option "time")
nonEffInterval	a numeric vector (a number of events or a number of weeks) specifying the timing of non-efficacy interim analyses. If a single numeric value is specified, then all interim looks are equidistant (in terms of the number of events or weeks), and the value specifies the constant increment of information or time between two adjacent interim looks. If a numeric vector with at least two components is specified, then, following the initial interim look, the timing of subsequent interim looks is determined by (potentially differential) increments of information or time specified by this vector.
nonEffCohorts	a named list specifying all cohorts (for both timing and analysis) used in non-efficacy monitoring. The required list components characterize the 'timing cohort,' i.e., the cohort events in which determine the analysis timepoints in an event-driven design (components times, timeUnit, and timingCohort), and the analysis cohort(s), i.e., the cohort(s) in which inference is made about the null hypothesis of non-efficacy (component cohort1 is required, and cohort2, etc. are optional for additional analysis cohorts if, e.g., non-efficacy monitoring is conducted in both the ITT and per-protocol cohorts). As for the timing cohort, times specifies the analysis timepoints in terms of event counts (\code{timeUnit} = \code{"counts"} is the only implemented option). timingCohort is a list characterizing the timing cohort by components lagTime, a lag time (in weeks) that controls event inclusion for timing (if no lag is desired, it can be set to NULL or 0), and cohortInd, a character string naming an indicator variable included in the data frames outputted by simTrial, which also controls event inclusion for the purpose of determining analysis timepoints. The other top-level list components cohort1, cohort2, etc. are each a list that must contain a component named estimand, which can take on values "cox", "cuminc", or "combined". Optional list components in cohort1, cohort2, etc. are lagTime, cohortInd, nullVE, and nominalAlphas. lagTime specifies a lag time (in weeks) that controls event inclusion for the analysis cohort. If no lag time is desired, then this component can be ignored, set to NULL, or set to 0. cohortInd is a character string naming an indicator variable included in the data frames outputted by simTrial to be used to subset participants into the analysis cohort. This component allows inclusion of, e.g., a "per-protocol" variable (e.g., by setting cohortInd to "pp1"). nullVE specifies the one-sided null hypothesis as H_0: VE ≥q \code{nullVE}. The rejection of H_0 constitutes evidence for non-efficacy. nominalAlphas specifies nominal significance levels in a two-arm event-driven trial design.
effCohort	a named list specifying the timing and analysis cohorts used in group-sequential efficacy monitoring (if part of the monitoring plan). The required list components characterize the 'timing cohort,' i.e., the cohort events in which determine the analysis timepoints in an event-driven design (components times, timeUnit, and timingCohort), and a single analysis cohort, i.e., the cohort in which inference is made about the null hypothesis of efficacy (components estimand, lagTime, cohortInd, nullVE, and nominalAlphas). As for the timing cohort, times specifies the analysis timepoints in terms of event counts (\code{timeUnit} = \code{"counts"} is the only implemented option). timingCohort is a list characterizing the timing cohort by components lagTime, a lag time (in weeks) that controls event inclusion for timing (if no lag is desired, it can be set to NULL or 0), and cohortInd, a character string naming an indicator variable included in the data frames outputted by simTrial, which also controls event inclusion for the purpose of determining analysis timepoints. As for the analysis cohort, estimand can take on values "cox", "cuminc", or "combined". lagTime specifies a lag time (in weeks) that controls event inclusion for the analysis cohort. If no lag time is desired, then this component can be ignored, set to NULL, or set to 0. cohortInd is a character string naming an indicator variable included in the data frames outputted by simTrial to be used to subset participants into the analysis cohort. This component allows inclusion of, e.g., a "per-protocol" variable (e.g., by setting cohortInd to "pp1"). nullVE specifies the one-sided primary null hypothesis as H_0: VE ≤q \code{nullVE}. The rejection of H_0 constitues evidence for clinically relevant efficacy. nominalAlphas specifies nominal significance levels in a two-arm event-driven trial design.
lowerVEnoneff	specifies an additional criterion for declaring non-efficacy if the hypothesis test is based on Wald confidence interval(s). It requires that the lower bound of the two-sided Wald CI(s) for the VE estimand(s), at the confidence level determined by nonEffCohorts$cohort1$nominalAlphas, etc., lie(s) below lowerVEnoneff (typically set equal to 0). If NULL (default), this criterion is ignored.
highVE	specifies a criterion for declaring high-efficacy: the lower bound of the two-sided (1-\code{alphaHigh}) 100\% confidence interval for the VE estimand lies above highVE (typically a number in the 0.5–1 range). To turn off high efficacy monitoring, set highVE equal to 1.
stage1VE	specifies a criterion for advancement of a treatment's evaluation into Stage 2: the lower bound of the two-sided (1-\code{alphaStage1}) 100\% confidence interval for the VE estimand lies above stage1VE (typically set equal to 0)
lowerVEuncPower	a numeric vector with each component specifying a one-sided null hypothesis H_0: VE(0-\code{stage1}) ≤q \code{lowerVEuncPower} \times 100\%. Unconditional power (i.e., accounting for sequential monitoring) to reject each H_0 is calculated, where the rejection region is defined by the lower bound of the two-sided (1-\code{alphaUncPower}) 100\% confidence interval for the VE estimand being above the respective component of lowerVEuncPower (typically values in the 0–0.5 range).
alphaHigh	one minus the nominal confidence level of the two-sided confidence interval used for high efficacy monitoring
alphaStage1	one minus the nominal confidence level of the two-sided confidence interval used for determining whether a treatment's evaluation advances into Stage 2
alphaUncPower	one minus the nominal confidence level of the two-sided confidence interval used to test one-sided null hypotheses H_0: VE(0-\code{stage1}) ≤q \code{lowerVEuncPower} \times 100\% against alternative hypotheses H_1: VE(0-\code{stage1}) > \code{lowerVEuncPower} \times 100\%. The same nominal confidence level is applied for each component of lowerVEuncPower.
saveFile	a character string specifying the name of the output .RData file. If NULL (default), a default file name will be used.
saveDir	a character string specifying a path for dataFile. If supplied, the output is also saved as an .RData file in this directory; otherwise the output is returned as a list.
verbose	a logical value indicating whether information on the output directory, file name, and monitoring outcomes should be printed out (default is TRUE)

Details

All time variables use week as the unit of time. Month is defined as 52/12 weeks.

Potential harm monitoring starts at the harmMonitorRange[1]-th infection pooled over the placebo group and the vaccine regimen that accrues infections the fastest. The potential harm analyses continue at each additional infection until the first interim analysis for non-efficacy. The monitoring is implemented with exact one-sided binomial tests of H0: p ≤ p0 versus H1: p > p0, where p is the probability that an infected participant was assigned to the vaccine group, and p0 is a fixed constant that represents the null hypothesis that an infection is equally likely to be assigned vaccine or placebo. Each test is performed at the same prespecified nominal/unadjusted alpha-level (alphaPerTest), chosen based on simulations such that, for each vaccine regimen, the overall type I error rate by the harmMonitorRange[2]-th arm-pooled infection (i.e., the probability that the potential harm boundary is reached when the vaccine is actually safe, p = p0) equals harmMonitorAlpha.

Non-efficacy is defined as evidence that it is highly unlikely that the vaccine has a beneficial effect measured as VE(0–stage1) of upperVEnoneff x 100% or more. The non-efficacy analyses for each vaccine regimen will start at the first infection (pooled over the vaccine and placebo arm) determined by nonEffStartMethod. Stopping for non-efficacy will lead to a reported two-sided (1-alphaNoneff) x 100% CI for VE(0–stage1) with, optionally, the lower confidence bound below lowerVEnoneff and the upper confidence bound below upperVEnoneff, where estimand determines the choice of the VE(0–stage1) estimand. This approach is similar to the inefficacy monitoring approach of Freidlin, Korn, and Gray (2010). If estimand = "combined", stopping for non-efficacy will lead to reported (1-alphaNoneff) x 100% CIs for both VE parameters with, optionally, lower confidence bounds below lowerVEnoneff and upper confidence bounds below upperVEnoneff. If laggedMonitoring = TRUE, stopping for non-efficacy will lead to reported (1-alphaNoneff) x 100% CIs for both VE(0–stage1) and VE(lagTime–stage1) with, optionally, lower confidence bounds below lowerVEnoneff and upper confidence bounds below upperVEnoneff.

High efficacy monitoring allows early detection of a highly protective vaccine if there is evidence that VE(0–stage2) > highVE x 100%. It is synchronized with non-efficacy monitoring during Stage 1, and a single high-efficacy interim analysis during Stage 2 is conducted halfway between the end of Stage 1 and the end of the trial. While monitoring for potential harm and non-efficacy restricts to stage1 infections, monitoring for high efficacy counts all infections during stage1 or stage2, given that early stopping for high efficacy would only be warranted under evidence for durability of the efficacy.

The following principles and rules are applied in the monitoring procedure:

• Exclude all follow-up data from the analysis post-unblinding (and include all data pre-unblinding).

• The monitoring is based on modified ITT analysis, i.e., all subjects documented to be free of the study endpoint at baseline are included and analyzed according to the treatment assigned by randomization, ignoring how many vaccinations they received (only pre-unblinding follow-up included).

• If a vaccine hits the harm boundary, immediately discontinue vaccinations and accrual into this vaccine arm, and unblind this vaccine arm (continue post-unblinded follow-up until the end of Stage 1 for this vaccine arm).

• If a vaccine hits the non-efficacy boundary, immediately discontinue vaccinations and accrual into this vaccine arm, keep blinded and continue follow-up until the end of Stage 1 for this vaccine arm.

• If and when the last vaccine arm hits the non-efficacy (or harm) boundary, discontinue vaccinations and accrual into this vaccine arm, and unblind (the trial is over, completed in Stage 1).

• Stage 1 for the whole trial is over on the earliest date of the two events: (1) all vaccine arms have hit the harm or non-efficacy boundary; and (2) the last enrolled subject in the trial reaches the final stage1 visit.

• Continue blinded follow-up until the end of Stage 2 for each vaccine arm that reaches the end of stage1 with a positive efficacy (as defined by stage1VE) or high efficacy (as defined by highVE) result.

• If at least one vaccine arm reaches the end of stage1 with a positive efficacy or high efficacy result, continue blinded follow-up in the placebo arm until the end of Stage 2.

• Stage 2 for the whole trial is over on the earliest date of the two events: (1) all subjects in the placebo arm and each vaccine arm that registered efficacy or high efficacy in stage1 have failed or been censored; and (2) all subjects in the placebo arm and each vaccine arm that registered efficacy or high efficacy in stage1 have completed the final stage2 visit.

The above rules have the following implications:

• If a vaccine hits the non-efficacy boundary but Stage 1 for the whole trial is not over, then one includes in the analysis all follow-up through the final stage1 visit for that vaccine regimen, including all individuals accrued up through the date of hitting the non-efficacy boundary (which will be the total number accrued to this vaccine arm).

• If a vaccine hits the harm boundary, all follow-up information through the date of hitting the harm boundary is included for this vaccine; no follow-up data are included after this date.

• If and when the last vaccine arm hits the non-efficacy (or harm) boundary, all follow-up information through the date of hitting the non-efficacy (or harm) boundary is included for this vaccine; no follow-up data are included after this date.

Value

If saveDir (and, optionally saveFile) is specified, the output list (named out) is saved as an .RData file in saveDir (the path to saveDir is printed); otherwise it is returned. The output object is a list of length equal to the number of simulated trials, each of which is a list of length equal to the number of treatment arms, each of which is a list with (at least) the following components:

• boundHit: a character string stating the monitoring outcome in this treatment arm, i.e., one of "Harm", "NonEffInterim", "NonEffFinal", "Eff", or "HighEff". The first four outcomes can occur in Stage 1, whereas the last outcome can combine data over Stage 1 and Stage 2.

• stopTime: the time of hitting a stopping boundary since the first subject enrolled in the trial

• stopInfectCnt: the pooled number of infections at stopTime

• summObj: a data.frame containing summary information from each non-/high efficacy interim analysis

• finalHRci: the final CI for the hazard ratio, available if estimand!="cuminc" and there is at least 1 infection in each arm

• firstNonEffCnt: the number of infections that triggered non-efficacy monitoring (if available)

• totInfecCnt: the total number of stage1 (stage2 if boundHit = "HighEff") infections

• totInfecSplit: a table with the numbers of stage1 (stage2 if boundHit = "HighEff") infections in the treatment and control arm

• lastExitTime: the time between the first subject's enrollment and the last subject's exiting from the trial

References

Freidlin B., Korn E. L., and Gray R. (2010), A general inefficacy interim monitoring rule for randomized clinical trials. Clinical Trials 7(3):197-208.

See Also

simTrial, censTrial, rankTrial, estHRbound, crossBoundCumProb, and decisionTimes

Examples

simData <- simTrial(N=c(1000, 1000), aveVE=c(0, 0.4),
                    VEmodel="half", vePeriods=c(1, 27, 79), enrollPeriod=78,
                    enrollPartial=13, enrollPartialRelRate=0.5, dropoutRate=0.05,
                    infecRate=0.06, fuTime=156, visitSchedule=seq(0, 156, by=4),
                    missVaccProb=0.05, VEcutoffWeek=26, nTrials=5,
                    stage1=78, randomSeed=300)
                    
### trial design: fixed follow-up; cumulative incidence-based VE estimand;
                  no efficacy monitoring; non-efficacy monitoring using 
                  the Freidlin et al. method; hypothesis tests based on Wald 
                  confidence intervals
### RIGHT NOW, THE BELOW CALL IS BROKEN
monitorData <- monitorTrial(dataFile=simData, stage1=78, stage2=156,
                            harmMonitorRange=c(10, NA), harmMonitorAlpha=0.05,
                            nonEffStartMethod="FKG", nonEffInterval=20,
                            nonEffCohorts=list(timeUnit="counts",
                                               # right now 'timingCohort' is required
                                               timingCohort=list(lagTime=0),
                                               cohort1=list(estimand="cuminc",
                                                            nullVE=0.4,
                                                            nominalAlphas=0.025)),
                            # it appears that right now 'effCohort' must be specified
                            # even when there is no efficacy monitoring;
                            # this call of monitorTrial() still doesn't run
                            # because it requires event counts for timing
                            effCohort=list(timeUnit="counts",
                                           timingCohort=list(lagTime=0)),
                            lowerVEnoneff=0, highVE=0.7, lowerVEuncPower=0, 
                            alphaHigh=0.05, alphaUncPower=0.05)
                            
### trial design: event-driven; hazard-based VE estimand;
                  harmonized efficacy and non-efficacy monitoring; 
                  hypothesis tests using the score test in the Cox model
### THE SCORE TEST OPTION NEEDS TO BE ADDED
monitorData <- monitorTrial(dataFile=simData, stage1=78, stage2=156,
                            harmMonitorRange=c(10, 50), harmMonitorAlpha=0.05,
                            nonEffCohorts=list(
                              times=c(50, 100, 150),
                              timeUnit="counts",
                              timingCohort=list(lagTime=0),
                              cohort1=list(estimand="cox",
                                           nullVE=0.4,
                                           nominalAlphas=c(0.0001, 0.0060, 0.0231))),
                            effCohort=list(times=c(50, 100, 150),
                                           timeUnit="counts",
                                           timingCohort=list(lagTime=0),
                                           estimand="cox",
                                           nullVE=0,
                                           nominalAlphas=c(0.0001, 0.0060, 0.0231)),
                            highVE=1, lowerVEuncPower=0, 
                            alphaHigh=0.05, alphaUncPower=0.05)
   
### alternatively, to save the .RData output file (no '<-' needed):
###
### simTrial(N=c(1000, 1000), aveVE=c(0, 0.4),
###          VEmodel="half", vePeriods=c(1, 27, 79), enrollPeriod=78,
###          enrollPartial=13, enrollPartialRelRate=0.5, dropoutRate=0.05,
###          infecRate=0.06, fuTime=156, visitSchedule=seq(0, 156, by=4),
###          missVaccProb=0.05, VEcutoffWeek=26, nTrials=5,
###          stage1=78, saveDir="./", randomSeed=300)
###
### THIS CALL NEEDS TO BE REVISED TO A SIMPLE BUT WORKING CODE
### THE INTENT HERE IS ONLY TO ILLUSTRATE HOW OUTPUT FILES CAN BE READ IN
### THE PURPOSE IS NOT TO SHOW ANY ADDITIONAL TRIAL DESIGNS
### monitorTrial(dataFile=
###              "simTrial_nPlac=1400_nVacc=1000_1000_aveVE=0.2_0.4_infRate=0.04.RData", 
###              stage1=78, stage2=156, harmMonitorRange=c(10,100), alphaPerTest=NULL, 
###              nonEffStartMethod="FKG", nonEffInterval=20, lowerVEnoneff=0, 
###              upperVEnoneff=0.4, highVE=0.7, stage1VE=0, lowerVEuncPower=0, 
###              alphaNoneff=0.05, alphaHigh=0.05, alphaStage1=0.05, alphaUncPower=0.05, 
###              estimand="cuminc", lagTime=26, saveDir="./")

mjuraska/seqDesign documentation built on Dec. 14, 2022, 4:26 p.m.