compute_design_performance: Compute Design Performance

Description Usage Arguments Details Value References Examples

Description

Generates decision rules for group sequential trial designs with adaptive enrollment criteria. Tables are also generated which compare the performance of these designs to the performance of standard group sequential designs. We use the notation AD to refer to the design with adaptive enrollment, SC to refer to a standard group sequential design enrolling from the combined population, and SS to refer to a standard group sequential design enrolling from only the subpopulation where there is greater prior evidence of a positive treatment effect. Further details are provided in (Fisher et al. 2014).

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
compute_design_performance(p_1 = 0.33, p_10 = 0.25, p_20 = 0.2,
  p_11 = p_10 + 0.125, per_stage_sample_size_combined_AD = 280,
  per_stage_sample_size_subpop_1_AD = 148, FWER = 0.025,
  FWER_H0C_proportion = 0.09, Delta = -0.5, iter = 10000,
  time_limit = 45, total_number_stages = 5,
  last_stage_subpop_2_enrolled_AD = 3,
  enrollment_rate_combined_population = 420,
  per_stage_sample_size_combined_SC = 106, per_stage_sample_size_SS = 100,
  subpop_2_stop_boundary_constant_AD = 0,
  H01_futility_boundary_constant_AD = 0,
  H0C_futility_boundary_constant_SC = -0.1,
  H01_futility_boundary_constant_SS = -0.1,
  lower_bound_treatment_effect_subpop_2 = -0.2,
  upper_bound_treatment_effect_subpop_2 = 0.2, CSV)

Arguments

p_1

The proportion of the population in subpopulation 1, which is the subpopulation having stronger prior evidence of a positive treatment effect.

p_10

The probability of a successful outcome for subpopulation 1 under assignment to the control arm. This is used in estimating power and expected sample size.

p_20

The probability of a successful outcome for subpopulation 2 under assignment to the control arm. This is used in estimating power and expected sample size.

p_11

The probability of a successful outcome for subpopulation 1 under assignment to the treatment arm. Note that the user does not specify the probability of success under treatment for subpopulation 2 (p_2t). Instead, compute_design_performance considers a range of possible values of p_2t (see the lower_bound_treatment_effect_subpop_2 and upper_bound_treatment_effect_subpop_2 arguments).

per_stage_sample_size_combined_AD

The number of participants enrolled per stage in the adaptive design while both subpopulations are being enrolled.

per_stage_sample_size_subpop_1_AD

The number of participants required for each stage in the adaptive design after enrollment for subpopulation 2 has been stopped.

FWER

The familywise Type I error rate (α) for all designs (see Fisher et al. 2014).

FWER_H0C_proportion

Proportion of α allocated to H_{0C} for the adaptive design. Here, H_{0C} refers to the null hypothesis of no treatment effect in the combined population.

Delta

Used as the exponent in defining the efficacy and futility boundaries, as described (Fisher et al. 2014).

iter

The number of simulated trials used to estimate the power, expected sample size, and expected trial duration.

time_limit

Time limit for simulation in seconds. If the simulation exceeds the time limit, calculations will stop and the user will get an error message ("reached CPU time limit"). See setTimeLimit. To avoid this, the number of iterations (iter) can be reduced or the time limit can be increased.

total_number_stages

Total number of stages used in each design (K). The maximum allowable number of stages is 20.

last_stage_subpop_2_enrolled_AD

The last stage subpopulation 2 is enrolled, under the adaptive design. We refer to this stage number as k*.

enrollment_rate_combined_population

The assumed enrollment rate per year for the combined population. This impacts the expected duration of each trial design. Active enrollments from the two subpopulations are assumed to be independent. The enrollment rates for subpopulations 1 and 2 are assumed proportional, based on p_1. This implies that each stage of the adaptive design up to and including stage k* takes the same amount of time to complete, regardless of whether or not enrollment stops for subpopulation 2. Each stage after k* will also take the same amount of time to complete.

per_stage_sample_size_combined_SC

The number of participants enrolled in each stage of the standard group sequential design enrolling the combined population (SC).

per_stage_sample_size_SS

The number of participants enrolled in each stage for standard group sequential design enrolling only subpopulation 1 (SS).

subpop_2_stop_boundary_constant_AD

Stopping boundary proportionality constant for subpopulation 2 enrollment in the adaptive design.

H01_futility_boundary_constant_AD

Futility boundary proportionality constant for H_{01} in the adaptive design. This is used to calculate the futility boundaries (l_{1,k}) for the z-statistics calculated in subpopulation 1 (Z_{1,k}) as defined in (Fisher et al. 2014).

H0C_futility_boundary_constant_SC

Futility boundary proportionality constant for H_{0C} in the standard design always enrolling from the combined population.

H01_futility_boundary_constant_SS

Futility boundary proportionality constant for H_{01} in the standard design only enrolling from subpopulation 1.

lower_bound_treatment_effect_subpop_2

Simulations are performed under a range of treatment effect sizes for subpopulation 2 (i.e. p_{2t}-p_{2c}). This parameter sets the lower bound for this range. This effectively sets the lower bound for the probability of success under treatment for subpopulation 2 (p_21), since p_20 is set by the user.

upper_bound_treatment_effect_subpop_2

Simulations are performed under a range of treatment effect sizes for subpopulation 2 (i.e. p_{2t}-p_{2c}). This parameter sets the upper bound for this range.

CSV

Rather than manually entering the arguments above, this allows for the arguments to be entered via a tabular csv file. The CSV argument should contain a character vector or list of csv filenames. The table must minimally include the columns "inputId" and "value" (as in the default_parameter_table).

Details

This function is meant to be applied when there is prior evidence that a treatment might work better in a one subpopulation than in another. In this context, a trial with an adaptive enrollment criteria would determine whether or not to continue enrolling patients from each subpopulation based on interim analyses of whether each subpopulation is benefiting. In order for the type I error and the power of the trial to be calculable, the decision rules for changing enrollment must be set before the trial starts. This function generates decision rules for group sequential trial designs with adaptive enrollment criteria, and compares the performance of these designs against standard group sequential designs with fixed enrollment criteria. Performance is compared in terms of power, expected sample size, and expected trial duration.

Value

A list with 5 elements:

performance_comparison

A table comparing the performance of the three trials, in terms of power, expected sample size, and expected duration. See examples.

AD_design

Efficacy and futility boundaries for the group sequential design with adaptive enrollment

SC_design

Efficacy and futility boundaries for the standard group sequential design enrolling from the combined population

SS_design

Efficacy and futility boundaries for the standard group sequential design enrolling subpopulation 1 only

input_parameters

List of input argument values

References

Aaron Fisher, Harris Jaffee, and Michael Rosenblum. interAdapt – An Interactive Tool for Designing and Evaluating Randomized Trials with Adaptive Enrollment Criteria. Working Paper, 2014. http://arxiv.org/abs/1404.0734

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
#Store function output
o<-compute_design_performance()

names(o)

#Decision rules for trial designs
print(o$SS[[1]])
print(o$SC[[1]])
print(o$AD[[1]])

#Plot decision rules
par(mfrow=c(1,3))
matplot(t(o$SS[[1]][2:3,]),type='o',
		main='Standard trial - subpop 1',
		xlab='stage',ylab='Z-statistic',
		col='blue',pch=2:3,lty=3)
legend('topright',c('H01 Efficacy','H01 Futility')
		,col='blue',pch=2:3,lty=3)
matplot(t(o$SC[[1]][4:5,]),type='o',
		main='Standard trial - combined pop',xlab='stage',
		ylab='Z-statistic',col='red',pch=0:1,lty=3)
legend('topright',c('H0C Efficacy','H0C Futility'),
		col='red',pch=0:1,lty=3)
matplot(t(o$AD[[1]][4:7,]),type='o',
		main='Adaptive Enrollment',xlab='stage',
		ylab='Z-statistic',col=c('red','red','blue','blue'),
		pch=0:3,lty=3)
legend('topright',c('H0C Efficacy','H0C Futility',
		'H01 Efficacy','H01 Futility'),col=c('red',
		'red','blue','blue'),pch=0:3,lty=3)

#Check performance
o$performance[[1]]


#Plot performance over a range of treatment effects for subpop2
col1<-c('black','black','black','green','blue')


perform_names<-rownames(o$performance[[1]])

#index for parts of the table corresponding to power
p_ind<- grep('Power',perform_names)
#index for parts of the table corresponding to sample size
s_ind<- grep('Sample Size',perform_names)
#index for parts of the table corresponding to trial duration
d_ind<- grep('DUR',perform_names)


par(mfrow=c(1,3))

lty1<-c(2,3,1,4,5)
matplot(x=t(o$performance[[1]][1,]),t(o$performance[[1]][p_ind,]),
		type='l',lty=lty1,col=col1[1:5],xlab='Subpop.2 Tx. Effect',
		ylab='Power',main='Power')
legend('bottomleft',perform_names[p_ind],col=col1[1:length(p_ind)],
		lty=lty1)


matplot(x=t(o$performance[[1]][1,]),t(o$performance[[1]][s_ind,]),
		type='l',lty=1:3,col=col1[3:5],xlab='Subpop.2 Tx. Effect',
		ylab='Expected Sample Size',main='Expected Sample Size')
legend('topright',substr(perform_names[d_ind],1,2),col=col1[3:5],
		lty=1:3)


matplot(x=t(o$performance[[1]][1,]),t(o$performance[[1]][d_ind,]),
		type='l',lty=1:3,col=col1[3:5],xlab='Subpop.2 Tx. Effect',
		ylab="Expected Duration",main='Expected Duration')
legend('topright',substr(perform_names[d_ind],1,2),col=col1[3:5],
		lty=1:3)

Example output

[1] "performance_comparison" "AD_design"              "SC_design"             
[4] "SS_design"              "input_parameters"      
                                                         1           2
Cumulative Sample Size                          100.000000 200.0000000
H01 Efficacy Boundaries for z-statistics Z(1,k)   4.559666   3.2241708
H01 Futility Boundaries for z-statistics Z(1,k)  -0.200000  -0.1414214
                                                          3          4
Cumulative Sample Size                          300.0000000 400.000000
H01 Efficacy Boundaries for z-statistics Z(1,k)   2.6325244   2.279833
H01 Futility Boundaries for z-statistics Z(1,k)  -0.1154701  -0.100000
                                                         5
Cumulative Sample Size                          500.000000
H01 Efficacy Boundaries for z-statistics Z(1,k)   2.039145
H01 Futility Boundaries for z-statistics Z(1,k)   2.039145
                                                         1           2
Cumulative Sample Size Subpop. 1                 34.980000  69.9600000
Cumulative Sample Size Subpop. 2                 71.020000 142.0400000
Cumulative Sample Size Combined Pop.            106.000000 212.0000000
H0C Efficacy Boundaries for z-statistics Z(C,k)   4.559666   3.2241708
H0C Futility Boundaries for z-statistics Z(C,k)  -0.200000  -0.1414214
                                                          3          4
Cumulative Sample Size Subpop. 1                104.9400000 139.920000
Cumulative Sample Size Subpop. 2                213.0600000 284.080000
Cumulative Sample Size Combined Pop.            318.0000000 424.000000
H0C Efficacy Boundaries for z-statistics Z(C,k)   2.6325244   2.279833
H0C Futility Boundaries for z-statistics Z(C,k)  -0.1154701  -0.100000
                                                         5
Cumulative Sample Size Subpop. 1                174.900000
Cumulative Sample Size Subpop. 2                355.100000
Cumulative Sample Size Combined Pop.            530.000000
H0C Efficacy Boundaries for z-statistics Z(C,k)   2.039145
H0C Futility Boundaries for z-statistics Z(C,k)   2.039145
                                                                   1          2
Cumulative Sample Size Subpop. 1                           92.400000 184.800000
Cumulative Sample Size Subpop. 2                          187.600000 375.200000
Cumulative Sample Size Combined Pop.                      280.000000 560.000000
H0C Efficacy Boundaries u(C,k) for z-statistics Z(C,k)      4.956000   3.504421
Boundaries l(2,k) for Z(2,k) to Stop Subpop. 2 Enrollment   0.000000   0.000000
H01 Efficacy Boundaries u(1,k) for Z(1,k)                   5.101976   3.607642
Boundaries l(1,k) for Z(1,k) to Stop All Enrollment         0.000000   0.000000
                                                                   3          4
Cumulative Sample Size Subpop. 1                          277.200000 425.200000
Cumulative Sample Size Subpop. 2                          562.800000 562.800000
Cumulative Sample Size Combined Pop.                      840.000000 988.000000
H0C Efficacy Boundaries u(C,k) for z-statistics Z(C,k)      2.861348         NA
Boundaries l(2,k) for Z(2,k) to Stop Subpop. 2 Enrollment        Inf         NA
H01 Efficacy Boundaries u(1,k) for Z(1,k)                   2.945628   2.378361
Boundaries l(1,k) for Z(1,k) to Stop All Enrollment         0.000000   0.000000
                                                                    5
Cumulative Sample Size Subpop. 1                           573.200000
Cumulative Sample Size Subpop. 2                           562.800000
Cumulative Sample Size Combined Pop.                      1136.000000
H0C Efficacy Boundaries u(C,k) for z-statistics Z(C,k)             NA
Boundaries l(2,k) for Z(2,k) to Stop Subpop. 2 Enrollment          NA
H01 Efficacy Boundaries u(1,k) for Z(1,k)                    2.048431
Boundaries l(1,k) for Z(1,k) to Stop All Enrollment          2.048431
                             X1          X2          X3           X4
Subpop.2 Tx. Effect  -0.2000000  -0.1555556  -0.1111111  -0.06666667
AD:Sample Size      573.9080000 578.3858800 577.8755600 598.50564000
AD:DUR                2.7872150   2.8192522   2.7956733   2.80145079
AD:Power H0C          0.0000000   0.0000000   0.0000000   0.04000000
AD:Power H01         81.5800000  82.2600000  81.4700000  82.39000000
AD:Power H0C or H01  81.5800000  82.2600000  81.4700000  82.39000000
SC:Sample Size      123.6384000 149.5024000 203.7214000 282.57480000
SC:DUR                0.2943771   0.3559581   0.4850510   0.67279714
SC:Power H0C          0.0000000   0.0300000   0.2000000   1.84000000
SS:Sample Size      358.7300000 357.0100000 358.0500000 356.70000000
SS:DUR                2.5882395   2.5758297   2.5833333   2.57359307
SS:Power H01         82.0700000  81.8400000  81.6700000  80.82000000
                              X5           X6           X7          X8
Subpop.2 Tx. Effect  -0.02222222   0.02222222   0.06666667   0.1111111
AD:Sample Size      666.04960000 760.70988000 778.39300000 703.1972000
AD:DUR                2.79216883   2.71581010   2.36965657   1.8502638
AD:Power H0C          1.52000000  13.61000000  43.45000000  73.8000000
AD:Power H01         81.13000000  75.30000000  52.97000000  23.9300000
AD:Power H0C or H01  81.49000000  82.15000000  83.65000000  86.1700000
SC:Sample Size      351.88820000 401.36900000 408.54520000 381.9922000
SC:DUR                0.83782905   0.95564048   0.97272667   0.9095052
SC:Power H0C         10.03000000  30.00000000  57.14000000  80.3700000
SS:Sample Size      359.11000000 359.36000000 356.60000000 358.5700000
SS:DUR                2.59098124   2.59278499   2.57287157   2.5870851
SS:Power H01         81.74000000  80.99000000  81.26000000  81.9300000
                             X9         X10
Subpop.2 Tx. Effect   0.1555556   0.2000000
AD:Sample Size      613.1808000 548.5851200
AD:DUR                1.4905867   1.3082294
AD:Power H0C         87.0900000  89.7500000
AD:Power H01          8.8200000   4.1600000
AD:Power H0C or H01  89.0500000  89.8500000
SC:Sample Size      340.7900000 303.8172000
SC:DUR                0.8114048   0.7233743
SC:Power H0C         93.2800000  98.2300000
SS:Sample Size      356.9500000 358.5400000
SS:DUR                2.5753968   2.5868687
SS:Power H01         82.1300000  81.0000000

interAdapt documentation built on May 2, 2019, 7:31 a.m.