Nothing
SAMprior for Continuous Endpoints
In SAMprior: Self-Adapting Mixture (SAM) Priors
library(SAMprior)
library(knitr)
knitr::opts_chunk$set(
fig.width = 1.62*4,
fig.height = 4
)
## setup up fast sampling when run on CRAN
is_CRAN <- !identical(Sys.getenv("NOT_CRAN"), "true")
## NOTE: for running this vignette locally, please uncomment the
## following line:
## is_CRAN <- FALSE
.user_mc_options <- list()
if (is_CRAN) {
.user_mc_options <- options(RBesT.MC.warmup=250, RBesT.MC.iter=500, RBesT.MC.chains=2, RBesT.MC.thin=1, RBesT.MC.control=list(adapt_delta=0.9))
}
Introduction
The self-adapting mixture prior (SAMprior) package is designed to enhance
the effectiveness and practicality of clinical trials by leveraging historical
information or real-world data [1]. The package incorporates historical data
into a new trial using an informative prior constructed based on historical
data while mixing a non-informative prior to enhance the robustness of
information borrowing. It utilizes a data-driven way to determine a self-adapting
mixture weight that dynamically favors the informative (non-informative) prior
component when there is little (substantial) evidence of prior-data conflict.
Operating characteristics are evaluated and compared to the robust
Meta-Analytic-Predictive (rMAP) prior [2], which assigns a fixed weight of 0.5.
SAM Prior Derivation
SAM prior is constructed by mixing an informative prior $\pi_1(\theta)$,
constructed based on historical data, with a non-informative prior
$\pi_0(\theta)$ using the mixture weight $w$ determined by SAM_weight
function according to the degree of prior-data conflict [1]. The following
sections describe how to construct SAM prior in details.
Informative Prior Construction based on Historical Data
We assume three historical data as follows:
set.seed(123)
std <- function(x) sd(x)/sqrt(length(x))
df_1 <- rnorm(40, 0, 3);
df_2 <- rnorm(50, 0, 3);
df_3 <- rnorm(60, 0, 3);
dat <- data.frame(study = c(1,2,3),
n = c(40, 50, 60),
mean = round(c(mean(df_1), mean(df_2), mean(df_3)), 3),
se = round(c(std(df_1), std(df_2), std(df_3)), 3))
kable(dat)
To construct informative priors based on the aforementioned three historical
data, we apply gMAP function from RBesT to perform meta-analysis. This
informative prior results in a representative form from a large MCMC samples,
and it can be converted to a parametric representation with the
automixfit function using expectation-maximization (EM) algorithm [3].
This informative prior is also called MAP prior.
sigma = 3
# load R packages
library(ggplot2)
theme_set(theme_bw()) # sets up plotting theme
set.seed(22)
map_mcmc <- gMAP(cbind(mean, se) ~ 1 | study,
weights=n,data=dat,
family=gaussian,
beta.prior=cbind(0, sigma),
tau.dist="HalfNormal",tau.prior=cbind(0,sigma/2))
map_automix <- automixfit(map_mcmc)
map_automix
plot(map_automix)$mix
The resulting MAP prior is approximated by a mixture of conjugate priors.
SAM Weight Determination
Let $\theta$ and $\theta_h$ denote the treatment effects associated with the
current arm data $D$ and historical $D_h$, respectively. Let $\delta$ denote
the clinically significant difference such that is $|\theta_h - \theta| \ge \delta$,
then $\theta_h$ is regarded as clinically distinct from $\theta$, and it is
therefore inappropriate to borrow any information from $D_h$. Consider two
hypotheses:
$$
H_0: \theta = \theta_h, ~~ H_1: \theta = \theta_h + \delta ~ \text{or} ~ \theta = \theta_h - \delta.
$$
$H_0$ represents that $D_h$ and $D$ are consistent (i.e., no prior-data
conflict) and thus information borrowing is desirable, whereas $H_1$ represents
that the treatment effect of $D$ differs from $D_h$ to such a degree that no
information should be borrowed.
The SAM prior uses the likelihood ratio test (LRT) statistics $R$ to quantify
the degree of prior-data conflict and determine the extent of information
borrowing.
$$
R = \frac{P(D | H_0, \theta_h)}{P(D | H_1, \theta_h)} = \frac{P(D | \theta = \theta_h)}{\max { P(D | \theta = \theta_h + \delta), P(D | \theta = \theta_h - \delta) }} ,
$$
where $P(D | \cdot)$ denotes the likelihood function. An alternative Bayesian
choice is the posterior probability ratio (PPR):
$$
R = \frac{P(D | H_0, \theta_h)}{P(D | H_1, \theta_h)} = \frac{P(H_0)}{P(H_1)} \times BF ,
$$
where $P(H_0)$ and $P(H_1)$ is the prior probabilities of $H_0$ and $H_1$
being true. $BF$ is the Bayes Factor that in this case is the same as LRT.
The SAM prior, denoted as $\pi_{\text{sam}}(\theta)$, is then defined as a mixture
of an informative prior $\pi_1(\theta)$, constructed based on $D_h$, with a
non-informative prior $\pi_0(\theta)$:
$$\pi_{\text{sam}}(\theta) = w \pi_1(\theta) + (1 - w) \pi_0(\theta)$$
where the mixture weight $w$ is calculated as:
$$w = \frac{R}{1 + R}.$$
As the level of prior-data conflict increases, the likelihood ratio $R$
decreases, resulting in a decrease in the weight $w$ assigned to the
informative prior and a decrease in information borrowing. As a result,
$\pi_{\text{sam}}(\theta)$ is data-driven and has the ability to self-adapt the
information borrowing based on the degree of prior-data conflict.
To calculate the SAM weight $w$, we first assume the sample size enrolled in
the control arm is $n = 30$, with $\theta = 0.4$ and $\sigma = 3$. Additionally,
we assume the effective size is $d = \frac{\theta - \theta_h}{\sigma} = 0.5$,
then we can apply
function SAM_weight in SAMprior as follows:
set.seed(234)
sigma <- 3 ## Standard deviation in the current trial
data.crt <- rnorm(35, mean = 0.4, sd = sigma)
wSAM_LRT <- SAM_weight(if.prior = map_automix,
delta = 0.5 * sigma,
data = data.crt)
cat('SAM weight: ', wSAM_LRT)
The default method to calculate $w$ is using LRT, which is fully data-driven.
However, if investigators want to incorporate prior information on prior-data
conflict to determine the mixture weight $w$, this can be achieved by using
PPR method as follows:
wSAM_PPR <- SAM_weight(if.prior = map_automix,
delta = 0.5 * sigma,
method.w = 'PPR',
prior.odds = 3/7,
data = data.crt)
cat('SAM weight: ', wSAM_PPR)
The prior.odds indicates the prior probability of $H_0$ over the prior
probability of $H_1$. In this case (e.g., prior.odds = 3/7), the prior
information favors the presence prior-data conflict and it results in a
decreased mixture weight.
When historical information is congruent with the current control arm, SAM
weight reaches to the highest peak. As the level of prior-data conflict
increases, SAM weight decreases. This demonstrates that SAM prior is data-driven
and self-adapting, favoring the informative (non-informative) prior component
when there is little (substantial) evidence of prior-data conflict.
weight_grid <- seq(-3, 3, by = 0.3)
weight_res <- lapply(weight_grid, function(x){
res <- c()
for(s in 1:300){
data.control <- rnorm(n = 35, mean = x, sd = sigma)
res <- c(res, SAM_weight(if.prior = map_automix,
delta = 0.5 * sigma,
data = data.control))
}
mean(res)
})
df_weight <- data.frame(grid = weight_grid,
weight = unlist(weight_res))
qplot(grid, weight, data = df_weight, geom = "line", main= "SAM Weight") +
xlab('Sample mean from control trial')+ ylab('Weight') +
geom_vline(xintercept = summary(map_automix)['mean'], linetype = 2, col = 'blue')
SAM Prior Construction
To construct the SAM prior, we mix the derived informative prior $\pi_1(\theta)$
with a vague prior $\pi_0(\theta)$ using pre-determined mixture weight by
SAM_prior function in SAMprior as follows:
unit_prior <- mixnorm(nf.prior = c(1, summary(map_automix)['mean'], sigma))
SAM.prior <- SAM_prior(if.prior = map_automix,
nf.prior = unit_prior,
weight = wSAM_LRT, sigma = sigma)
SAM.prior
where the non-informative prior $\pi_0(\theta)$ follows an unit-information prior.
Operating Characteristics
In this section, we aim to investigate the operating characteristics of
the SAM prior, constructed based on the historical data.
The incorporation of historical information is expected to be beneficial
in reducing the required sample size for the current arms. To achieve this,
we assume a 1:2 ratio between the control and treatment arms.
We compare the operating characteristics of the SAM prior and rMAP prior
with pre-specified fixed weight under various scenarios. Specifically,
we will evaluate the relative bias and relative mean squared error (MSE)
of these methods. The relative bias and relative MSE are defined
as the differences between the bias/MSE of a given method and the bias/MSE
obtained when using a non-informative prior for the estimated effect in the
control arm.
Additionally, we investigate the Type I error and power of the methods
under different degrees of prior-data conflicts. The decision regarding
whether a treatment is superior or inferior to a standard control will be
based on the condition:
$$\Pr(\theta_t - \theta > 0 \mid D) > C,$$
where $\Delta$ is the clinical margin, and we let $\Delta = 0$.
$C$ is the posterior probability cutoff. We calibrated the posterior
probability cutoff under the null hypothesis, where there is no treatment
effect difference between the treatment and control arms and the historical
data and current control arm are assumed to arise from the same data-generating
process, to ensure the Type I error is maintained at the nominal level.
In SAMprior, the operating characteristics can be considered in following steps:
-
Specify priors: This step involves constructing informative prior based
on historical data and non-informative prior.
-
Specify design parameters for the get_OC function: This step
involves defining the design parameters for evaluating the operating
characteristics. These parameters include the clinically significant
difference (CSD) used in SAM prior calculation, the method used to
determine the mixture weight for the SAM prior, the sample sizes for
the control and treatment arms, the choice of weight for the robust MAP
prior used as a benchmark.
-
Scenarios: This step specifies the vectors of response rates for the
control and treatment arms. The get_OC function calibrates the
posterior probability cutoff under the assumption that the null calibration
scenario is given by $\theta =$ theta[1] and
$\theta_t =$ theta[1] $+ \Delta$ with $\Delta = 0$.
Type I Error
To evaluate Type I error, we consider four scenarios. In the first scenario,
we assume $\theta = \theta_t = \theta_h$, so each method should calibrate
the Type I error rate to the nominal level. The second and third scenarios
represent minimal prior-data conflict, whereas the last scenario represents
substantial prior-data conflict. Overall, the results indicate that both
SAM and rMAP effectively control the Type I error under minimal prior-data
conflict, while SAM demonstrates better Type I error control in the presence
of substantial prior-data conflict.
# weak_prior <- mixnorm(c(1, summary(map_automix)[1], 1e4))
TypeI <- get_OC(if.prior = map_automix, ## MAP prior from historical data
nf.prior = unit_prior, ## Unit-informative prior for mixture prior
prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm
delta = 0.5*sigma, ## CSD for SAM prior
## Method to determine the mixture weight for the SAM prior
method.w = 'LRT',
n = 35, n.t = 70, ## Sample size for control and treatment arms
if.rMAP = TRUE, ## Output robust MAP prior for comparison
weight = 0.5, ## Weight for robust MAP prior
## Trial settings
alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less"
margin = 0, ## Clinical margin
## Treatment effect for control and treatment arms
theta = c(summary(map_automix)['mean'], 0, -0.2, 2),
theta.t = c(summary(map_automix)['mean'], -0.1, -0.2, 2))
kable(TypeI)
Power
For power evaluation, we also consider four scenarios. In the first scenario,
we assume $\theta = \theta_t = \theta_h$, so each method should calibrate the
Type I error rate to the nominal level. The second and third scenarios represent
minimal prior-data conflict, whereas the last scenario represents substantial
prior-data conflict. Overall, the results show that the SAM prior achieves
performance similar to that of rMAP, and both outperform non-informative prior
(NP) when prior-data conflict is minimal. However, when prior-data conflict is
substantial, the SAM prior yields better performance than rMAP.
Power <- get_OC(if.prior = map_automix, ## MAP prior from historical data
nf.prior = unit_prior, ## Unit-information prior for mixture prior
prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm
delta = 0.5 * sigma, ## CSD for SAM prior
## Method to determine the mixture weight for the SAM prior
method.w = 'LRT',
n = 35, n.t = 70, ## Sample size for control and treatment arms
if.rMAP = TRUE, ## Output robust MAP prior for comparison
weight = 0.5, ## Weight for robust MAP prior
## Trial settings
alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less"
margin = 0, ## Clinical margin
## Treatment effect for control and treatment arms
theta = c(summary(map_automix)['mean'], 0.1, 0.5, -2),
theta.t = c(summary(map_automix)['mean'], 1.1, 2.0, -0.5))
kable(Power)
Decision Making
Finally, we present an example showing how to calibrate the posterior
probability cutoff and make a final decision on whether the treatment
is superior to a standard control after the trial has been
completed and the data have been collected.
The calibration step aims to identify the posterior probability cutoff $C$
such that the Type I error is controlled at a prespecified level.
For superiority, this is based on the posterior decision rule
$$
\Pr(\theta_t - \theta > \Delta \mid D) > C,
$$
where $\Delta$ is the clinical margin, which is often 0.
This calibration can be carried out using the calibrate_cutoff_2arm
function, as illustrated below:
## Calibrate the posterior probability cutoff
PPC <- calibrate_cutoff_2arm(if.prior = map_automix, ## MAP prior from historical data
nf.prior = unit_prior, ## Unit-information prior for mixture prior
prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm
target = 0.05, ## Targeted Type I error rate
n.t = 70, n = 35, ## Sample size for treatment and control arms, respectively
theta.t = summary(map_automix)['mean'], ## The true effect for treatment arm
theta = summary(map_automix)['mean'], ## The true effect for control arm
sigma.t = sigma, sigma = sigma, ## Standard deviation in the treatment and control arms, respectively
## Method to determine the mixture weight for the SAM prior
method = 'SAM', ## Method
delta = 0.2, ## CSD for SAM prior
## Trial settings
alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less".
margin = 0. ## Clinical margin.
)
cat("Calibrated posterior probability cutoff:", round(PPC$cutoff, 4), "\n")
To make the final decision using the calibrated posterior probability cutoff
$C$, we next update the posterior distributions based on the observed trial
data. In this example, the final posterior updating step is carried out using
the postmix function from RBesT, while the calibrated cutoff $C$ is
obtained from calibrate_cutoff_2arm in SAMprior. The treatment is then
declared superior if the resulting posterior probability exceeds the calibrated
cutoff. The posterior updating step is illustrated below:
## Simulate data for treatment arm
data.trt <- rnorm(70, mean = 3, sd = sigma)
## first obtain posterior distributions...
post_SAM <- postmix(priormix = SAM.prior, ## SAM Prior
data = data.crt)
post_trt <- postmix(priormix = unit_prior, ## Non-informative prior
data = data.trt)
## Define the decision function
decision = decision2S(PPC$cutoff, 0, lower.tail=FALSE)
## Decision-making
decision(post_trt, post_SAM)
References
[1] Yang P. et al., Biometrics, 2023; 00, 1–12. https://doi.org/10.1111/biom.13927 \
[2] Schmidli H. et al., Biometrics, 2014; 70(4):1023-1032. \
[3] Neuenschwander B. et al., Clin Trials, 2010; 7(1):5-18.
R Session Info
sessionInfo()
options(.user_mc_options)
Try the SAMprior package in your browser
Any scripts or data that you put into this service are public.
SAMprior documentation built on April 28, 2026, 1:07 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
library(SAMprior) library(knitr) knitr::opts_chunk$set( fig.width = 1.62*4, fig.height = 4 ) ## setup up fast sampling when run on CRAN is_CRAN <- !identical(Sys.getenv("NOT_CRAN"), "true") ## NOTE: for running this vignette locally, please uncomment the ## following line: ## is_CRAN <- FALSE .user_mc_options <- list() if (is_CRAN) { .user_mc_options <- options(RBesT.MC.warmup=250, RBesT.MC.iter=500, RBesT.MC.chains=2, RBesT.MC.thin=1, RBesT.MC.control=list(adapt_delta=0.9)) }
Introduction
The self-adapting mixture prior (SAMprior) package is designed to enhance the effectiveness and practicality of clinical trials by leveraging historical information or real-world data [1]. The package incorporates historical data into a new trial using an informative prior constructed based on historical data while mixing a non-informative prior to enhance the robustness of information borrowing. It utilizes a data-driven way to determine a self-adapting mixture weight that dynamically favors the informative (non-informative) prior component when there is little (substantial) evidence of prior-data conflict. Operating characteristics are evaluated and compared to the robust Meta-Analytic-Predictive (rMAP) prior [2], which assigns a fixed weight of 0.5.
SAM Prior Derivation
SAM prior is constructed by mixing an informative prior $\pi_1(\theta)$,
constructed based on historical data, with a non-informative prior
$\pi_0(\theta)$ using the mixture weight $w$ determined by SAM_weight
function according to the degree of prior-data conflict [1]. The following
sections describe how to construct SAM prior in details.
Informative Prior Construction based on Historical Data
We assume three historical data as follows:
set.seed(123) std <- function(x) sd(x)/sqrt(length(x)) df_1 <- rnorm(40, 0, 3); df_2 <- rnorm(50, 0, 3); df_3 <- rnorm(60, 0, 3); dat <- data.frame(study = c(1,2,3), n = c(40, 50, 60), mean = round(c(mean(df_1), mean(df_2), mean(df_3)), 3), se = round(c(std(df_1), std(df_2), std(df_3)), 3)) kable(dat)
To construct informative priors based on the aforementioned three historical
data, we apply gMAP function from RBesT to perform meta-analysis. This
informative prior results in a representative form from a large MCMC samples,
and it can be converted to a parametric representation with the
automixfit function using expectation-maximization (EM) algorithm [3].
This informative prior is also called MAP prior.
sigma = 3 # load R packages library(ggplot2) theme_set(theme_bw()) # sets up plotting theme set.seed(22) map_mcmc <- gMAP(cbind(mean, se) ~ 1 | study, weights=n,data=dat, family=gaussian, beta.prior=cbind(0, sigma), tau.dist="HalfNormal",tau.prior=cbind(0,sigma/2)) map_automix <- automixfit(map_mcmc) map_automix plot(map_automix)$mix
The resulting MAP prior is approximated by a mixture of conjugate priors.
SAM Weight Determination
Let $\theta$ and $\theta_h$ denote the treatment effects associated with the current arm data $D$ and historical $D_h$, respectively. Let $\delta$ denote the clinically significant difference such that is $|\theta_h - \theta| \ge \delta$, then $\theta_h$ is regarded as clinically distinct from $\theta$, and it is therefore inappropriate to borrow any information from $D_h$. Consider two hypotheses:
$$ H_0: \theta = \theta_h, ~~ H_1: \theta = \theta_h + \delta ~ \text{or} ~ \theta = \theta_h - \delta. $$ $H_0$ represents that $D_h$ and $D$ are consistent (i.e., no prior-data conflict) and thus information borrowing is desirable, whereas $H_1$ represents that the treatment effect of $D$ differs from $D_h$ to such a degree that no information should be borrowed.
The SAM prior uses the likelihood ratio test (LRT) statistics $R$ to quantify the degree of prior-data conflict and determine the extent of information borrowing. $$ R = \frac{P(D | H_0, \theta_h)}{P(D | H_1, \theta_h)} = \frac{P(D | \theta = \theta_h)}{\max { P(D | \theta = \theta_h + \delta), P(D | \theta = \theta_h - \delta) }} , $$ where $P(D | \cdot)$ denotes the likelihood function. An alternative Bayesian choice is the posterior probability ratio (PPR): $$ R = \frac{P(D | H_0, \theta_h)}{P(D | H_1, \theta_h)} = \frac{P(H_0)}{P(H_1)} \times BF , $$ where $P(H_0)$ and $P(H_1)$ is the prior probabilities of $H_0$ and $H_1$ being true. $BF$ is the Bayes Factor that in this case is the same as LRT.
The SAM prior, denoted as $\pi_{\text{sam}}(\theta)$, is then defined as a mixture of an informative prior $\pi_1(\theta)$, constructed based on $D_h$, with a non-informative prior $\pi_0(\theta)$: $$\pi_{\text{sam}}(\theta) = w \pi_1(\theta) + (1 - w) \pi_0(\theta)$$ where the mixture weight $w$ is calculated as: $$w = \frac{R}{1 + R}.$$ As the level of prior-data conflict increases, the likelihood ratio $R$ decreases, resulting in a decrease in the weight $w$ assigned to the informative prior and a decrease in information borrowing. As a result, $\pi_{\text{sam}}(\theta)$ is data-driven and has the ability to self-adapt the information borrowing based on the degree of prior-data conflict.
To calculate the SAM weight $w$, we first assume the sample size enrolled in
the control arm is $n = 30$, with $\theta = 0.4$ and $\sigma = 3$. Additionally,
we assume the effective size is $d = \frac{\theta - \theta_h}{\sigma} = 0.5$,
then we can apply
function SAM_weight in SAMprior as follows:
set.seed(234) sigma <- 3 ## Standard deviation in the current trial data.crt <- rnorm(35, mean = 0.4, sd = sigma) wSAM_LRT <- SAM_weight(if.prior = map_automix, delta = 0.5 * sigma, data = data.crt) cat('SAM weight: ', wSAM_LRT)
The default method to calculate $w$ is using LRT, which is fully data-driven. However, if investigators want to incorporate prior information on prior-data conflict to determine the mixture weight $w$, this can be achieved by using PPR method as follows:
wSAM_PPR <- SAM_weight(if.prior = map_automix, delta = 0.5 * sigma, method.w = 'PPR', prior.odds = 3/7, data = data.crt) cat('SAM weight: ', wSAM_PPR)
The prior.odds indicates the prior probability of $H_0$ over the prior
probability of $H_1$. In this case (e.g., prior.odds = 3/7), the prior
information favors the presence prior-data conflict and it results in a
decreased mixture weight.
When historical information is congruent with the current control arm, SAM weight reaches to the highest peak. As the level of prior-data conflict increases, SAM weight decreases. This demonstrates that SAM prior is data-driven and self-adapting, favoring the informative (non-informative) prior component when there is little (substantial) evidence of prior-data conflict.
weight_grid <- seq(-3, 3, by = 0.3) weight_res <- lapply(weight_grid, function(x){ res <- c() for(s in 1:300){ data.control <- rnorm(n = 35, mean = x, sd = sigma) res <- c(res, SAM_weight(if.prior = map_automix, delta = 0.5 * sigma, data = data.control)) } mean(res) }) df_weight <- data.frame(grid = weight_grid, weight = unlist(weight_res)) qplot(grid, weight, data = df_weight, geom = "line", main= "SAM Weight") + xlab('Sample mean from control trial')+ ylab('Weight') + geom_vline(xintercept = summary(map_automix)['mean'], linetype = 2, col = 'blue')
SAM Prior Construction
To construct the SAM prior, we mix the derived informative prior $\pi_1(\theta)$
with a vague prior $\pi_0(\theta)$ using pre-determined mixture weight by
SAM_prior function in SAMprior as follows:
unit_prior <- mixnorm(nf.prior = c(1, summary(map_automix)['mean'], sigma)) SAM.prior <- SAM_prior(if.prior = map_automix, nf.prior = unit_prior, weight = wSAM_LRT, sigma = sigma) SAM.prior
where the non-informative prior $\pi_0(\theta)$ follows an unit-information prior.
Operating Characteristics
In this section, we aim to investigate the operating characteristics of the SAM prior, constructed based on the historical data. The incorporation of historical information is expected to be beneficial in reducing the required sample size for the current arms. To achieve this, we assume a 1:2 ratio between the control and treatment arms.
We compare the operating characteristics of the SAM prior and rMAP prior with pre-specified fixed weight under various scenarios. Specifically, we will evaluate the relative bias and relative mean squared error (MSE) of these methods. The relative bias and relative MSE are defined as the differences between the bias/MSE of a given method and the bias/MSE obtained when using a non-informative prior for the estimated effect in the control arm.
Additionally, we investigate the Type I error and power of the methods under different degrees of prior-data conflicts. The decision regarding whether a treatment is superior or inferior to a standard control will be based on the condition: $$\Pr(\theta_t - \theta > 0 \mid D) > C,$$ where $\Delta$ is the clinical margin, and we let $\Delta = 0$. $C$ is the posterior probability cutoff. We calibrated the posterior probability cutoff under the null hypothesis, where there is no treatment effect difference between the treatment and control arms and the historical data and current control arm are assumed to arise from the same data-generating process, to ensure the Type I error is maintained at the nominal level.
In SAMprior, the operating characteristics can be considered in following steps:
-
Specify priors: This step involves constructing informative prior based on historical data and non-informative prior.
-
Specify design parameters for the
get_OCfunction: This step involves defining the design parameters for evaluating the operating characteristics. These parameters include the clinically significant difference (CSD) used in SAM prior calculation, the method used to determine the mixture weight for the SAM prior, the sample sizes for the control and treatment arms, the choice of weight for the robust MAP prior used as a benchmark. -
Scenarios: This step specifies the vectors of response rates for the control and treatment arms. The
get_OCfunction calibrates the posterior probability cutoff under the assumption that the null calibration scenario is given by $\theta =$theta[1]and $\theta_t =$theta[1]$+ \Delta$ with $\Delta = 0$.
Type I Error
To evaluate Type I error, we consider four scenarios. In the first scenario, we assume $\theta = \theta_t = \theta_h$, so each method should calibrate the Type I error rate to the nominal level. The second and third scenarios represent minimal prior-data conflict, whereas the last scenario represents substantial prior-data conflict. Overall, the results indicate that both SAM and rMAP effectively control the Type I error under minimal prior-data conflict, while SAM demonstrates better Type I error control in the presence of substantial prior-data conflict.
# weak_prior <- mixnorm(c(1, summary(map_automix)[1], 1e4)) TypeI <- get_OC(if.prior = map_automix, ## MAP prior from historical data nf.prior = unit_prior, ## Unit-informative prior for mixture prior prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm delta = 0.5*sigma, ## CSD for SAM prior ## Method to determine the mixture weight for the SAM prior method.w = 'LRT', n = 35, n.t = 70, ## Sample size for control and treatment arms if.rMAP = TRUE, ## Output robust MAP prior for comparison weight = 0.5, ## Weight for robust MAP prior ## Trial settings alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less" margin = 0, ## Clinical margin ## Treatment effect for control and treatment arms theta = c(summary(map_automix)['mean'], 0, -0.2, 2), theta.t = c(summary(map_automix)['mean'], -0.1, -0.2, 2)) kable(TypeI)
Power
For power evaluation, we also consider four scenarios. In the first scenario, we assume $\theta = \theta_t = \theta_h$, so each method should calibrate the Type I error rate to the nominal level. The second and third scenarios represent minimal prior-data conflict, whereas the last scenario represents substantial prior-data conflict. Overall, the results show that the SAM prior achieves performance similar to that of rMAP, and both outperform non-informative prior (NP) when prior-data conflict is minimal. However, when prior-data conflict is substantial, the SAM prior yields better performance than rMAP.
Power <- get_OC(if.prior = map_automix, ## MAP prior from historical data nf.prior = unit_prior, ## Unit-information prior for mixture prior prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm delta = 0.5 * sigma, ## CSD for SAM prior ## Method to determine the mixture weight for the SAM prior method.w = 'LRT', n = 35, n.t = 70, ## Sample size for control and treatment arms if.rMAP = TRUE, ## Output robust MAP prior for comparison weight = 0.5, ## Weight for robust MAP prior ## Trial settings alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less" margin = 0, ## Clinical margin ## Treatment effect for control and treatment arms theta = c(summary(map_automix)['mean'], 0.1, 0.5, -2), theta.t = c(summary(map_automix)['mean'], 1.1, 2.0, -0.5)) kable(Power)
Decision Making
Finally, we present an example showing how to calibrate the posterior probability cutoff and make a final decision on whether the treatment is superior to a standard control after the trial has been completed and the data have been collected.
The calibration step aims to identify the posterior probability cutoff $C$ such that the Type I error is controlled at a prespecified level. For superiority, this is based on the posterior decision rule $$ \Pr(\theta_t - \theta > \Delta \mid D) > C, $$ where $\Delta$ is the clinical margin, which is often 0.
This calibration can be carried out using the calibrate_cutoff_2arm
function, as illustrated below:
## Calibrate the posterior probability cutoff PPC <- calibrate_cutoff_2arm(if.prior = map_automix, ## MAP prior from historical data nf.prior = unit_prior, ## Unit-information prior for mixture prior prior.t = mixnorm(c(1,0,1000)), ## Vague prior for treatment arm target = 0.05, ## Targeted Type I error rate n.t = 70, n = 35, ## Sample size for treatment and control arms, respectively theta.t = summary(map_automix)['mean'], ## The true effect for treatment arm theta = summary(map_automix)['mean'], ## The true effect for control arm sigma.t = sigma, sigma = sigma, ## Standard deviation in the treatment and control arms, respectively ## Method to determine the mixture weight for the SAM prior method = 'SAM', ## Method delta = 0.2, ## CSD for SAM prior ## Trial settings alternative = "greater", ## Direction of the posterior decision. Must be one of "greater" or "less". margin = 0. ## Clinical margin. )
cat("Calibrated posterior probability cutoff:", round(PPC$cutoff, 4), "\n")
To make the final decision using the calibrated posterior probability cutoff
$C$, we next update the posterior distributions based on the observed trial
data. In this example, the final posterior updating step is carried out using
the postmix function from RBesT, while the calibrated cutoff $C$ is
obtained from calibrate_cutoff_2arm in SAMprior. The treatment is then
declared superior if the resulting posterior probability exceeds the calibrated
cutoff. The posterior updating step is illustrated below:
## Simulate data for treatment arm data.trt <- rnorm(70, mean = 3, sd = sigma) ## first obtain posterior distributions... post_SAM <- postmix(priormix = SAM.prior, ## SAM Prior data = data.crt) post_trt <- postmix(priormix = unit_prior, ## Non-informative prior data = data.trt) ## Define the decision function decision = decision2S(PPC$cutoff, 0, lower.tail=FALSE) ## Decision-making decision(post_trt, post_SAM)
References
[1] Yang P. et al., Biometrics, 2023; 00, 1–12. https://doi.org/10.1111/biom.13927 \ [2] Schmidli H. et al., Biometrics, 2014; 70(4):1023-1032. \ [3] Neuenschwander B. et al., Clin Trials, 2010; 7(1):5-18.
R Session Info
sessionInfo()
options(.user_mc_options)
Try the SAMprior package in your browser
Any scripts or data that you put into this service are public.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.