The costs of sampling each additional unit in (multilevel) experimental studies vary across treatment conditions (and levels of hierarchy). This package is a tool to identify the optimal sample allocation in experimental studies such that statistical power is maximized under a fixed budget. The designs covers experiments detecting main, mediation, and moderation effects (and some of the combinations).
Optimal design parameters in an optimal sample allocation include
Researchers can fix one or more of the optimal design parameters for constrained optimal sample allocation and solve for the remainder of optimal design parameters.
df = data.frame( Design = c("Simple Experiments", "2-Level CRTs", "2-Level MRTs", "3-Level CRTs", "3-Level MRTs", "4-Level CRTs", "4-Level MRTs"), OD_Function = c("od.1", "od.2", "od.2m", "od.3", "od.3m", "od.4", "od.4m"), Power_Function = c("power.1", "power.2", "power.2m", "power.3", "power.3m", "power.4", "power.4m"), RE_Function = c("re", "re", "re", "re", "re", "re", "re") ) names(df) <- c("Design", "OD Function", "Power Function", "RE Function") library(knitr) kable(df, caption = "Functions for Designs Detecting Main Effects")
For experiments detecting main effects, this package includes three categorical of functions and they are od, power, and re. For od and power functions, if a function name follows .# (e.g., od.2), this function is for #-level designs with treatment assignment at the top level (# in 1, 2, 3, 4). If a function name follows .#m (e.g., od.2m), this function is for #-level multisite-randomized trials.
This function calculates optimal sample allocation parameters with and without constraint(s). For each type of multilevel experimental studies, there is an additional number (and one additional letter m) to be added to the general function name. For example, the function for two-level cluster randomized trials is od.2, the function for two-level multisite randomized trials is od.2m.
This function performs power analyses with and without accommodating cost structures of sampling. For power analysis accommodating cost structures, depending on which one parameter is left, the function calculates required budget (and sample size), statistical power, and minimum detectable effect size. For conventional power analysis without accommodating cost structures of sampling, this function calculates required sample size, statistical power, and minimum detectable effect size. For each type of experiments, there is an additional number (and an additional letter m) to be added to the general function name. For example, the function for two-level cluster randomized trials is power.2, the function for two-level multisite randomized trials is power.2m.
This function calculates relative efficiency values between two designs. An alternative name for this function is rpe that stands for "relative precision and efficiency".
df = data.frame( Design = c("Simple Experiments", "2-Level CRTs", "2-Level MRTs"), OD_Function = c("od.1.111", "od.2.221", "od.2m.111"), Power_Function = c("power.1.111", "power.2.221", "power.2m.111") ) names(df) <- c("Design", "OD Function", "Power Function") library(knitr) kable(df, caption = "Functions for Designs Detecting Mediation Effects (In Preparation)")
There are two categories of functions for designs detecting mediation effects. They are od and power functions. The function names follow additional rules as these functions will add additional numbers to represent the levels of the treatment assignment, the mediator and the outcome. For example, od.2.221 is the od function for two-level cluster-randomized trials (.2) with the treatment assignment and the mediator at the level two, and the outcome at the level 1 (.221). od.2m.111 is the od function for two-level multisite-randomized trials (.2m) with the treatment assignment, the mediator, and the outcome at the level one (.111).
There are two categorical functions for designs detecting mediation effects and they are od and power. od function can calculate optimal design parameters with and without a constraint(s). power function can perform power analysis.
Given cost structure (i.e., the costs of sampling each unit at different levels and treatment conditions), this function solves the optimal sample allocation with and without constraints.
To solve the optimal sample allocation of a two-level cluster-randomized trial, we need the following information
library(odr)
# unconstrained optimal design myod1 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, varlim = c(0.01, 0.02)) # The function by default prints messages of output and plots the variance curves; one can turn off message and specify one or no plot. # myod1$out # output; # myod1$par # parameters used in the calculation.
# constrained optimal design with n = 20 myod2 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, plot.by = list(p = "p"), n = 20, varlim = c(0.005, 0.030)) # myod2$out # output # myod2$par # parameters used in the calculation.
# constrained optimal design with p = 0.5 myod3 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, p = 0.5, varlim = c(0.005, 0.020)) # myod3$out # output; # myod3$par # parameters used in the calculation.
# constrained n and p, no calculation performed myod4 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, plots = FALSE, n = 20, p = 0.5, varlim = c(0.005, 0.025))
Please see additional examples in corresponding functions by uncommenting below lines.
# ?od.1 # ?od.3 # ?od.4 # ?od.2m # ?od.3m # ?od.4m
This function by default can perform power analyses accommodating cost structures (i.e., cost.model = TRUE), one of 'power', 'm', and 'd' must be NULL. For example, if 'power' is NULL, the function calculates statistical power under a fixed budget and cost structure; if 'd' is NULL, the function calculates minimum detectable effect size (i.e., d) under a fixed budget and desired power level; if 'm' is NULL, the function calculate required budget (and required sample size) to achieve desired power level to detect a treatment effect.
This function also can conduct conventional power analysis or power analysis without accommodating cost structures by specifying cost.model = FALSE, the conventional power analyses include statistical power calculation, minimum detectable effect size calculation, and required sample size calculation.
mym <- power.2(expr = myod1, d = 0.3, q = 1, power = 0.8) # mym$out # m =1702, J = 59
figure <- par(mfrow = c(1, 2)) budget <- NULL nrange <- c(2:50) for (n in nrange) budget <- c(budget, power.2(expr = myod1, constraint = list (n = n), d = 0.3, q = 1, power = 0.8)$out$m) plot(nrange, budget, type = "l", lty = 1, xlim = c(0, 50), ylim = c(1500, 3500), xlab = "Level-1 sample size: n", ylab = "Budget", main = "", col = "black") abline(v = 9, lty = 2, col = "Blue") budget <- NULL prange <- seq(0.05, 0.95, by = 0.005) for (p in prange) budget <- c(budget, power.2(expr = myod1, constraint = list (p = p), d = 0.3, q = 1, power = 0.8)$out$m) plot(prange, budget, type = "l", lty = 1, xlim = c(0, 1), ylim = c(1500, 7000), xlab = "Porportion groups in treatment: p", ylab = "Budget", main = "", col = "black") abline(v = 0.33, lty = 2, col = "Blue") par(figure)
mypower <- power.2(expr = myod1, q = 1, d = 0.3, m = 1702) # mypower$out # power = 0.80
figure <- par(mfrow = c (1, 2)) pwr <- NULL nrange <- c(2:50) for (n in nrange) pwr <- c(pwr, power.2(expr = myod1, constraint = list (n = n), d = 0.3, q = 1, m = 1702)$out) plot(nrange, pwr, type = "l", lty = 1, xlim = c(0, 50), ylim = c(0.4, 0.9), xlab = "Level-1 sample size: n", ylab = "Power", main = "", col = "black") abline(v = 9, lty = 2, col = "Blue") pwr <- NULL prange <- seq(0.05, 0.95, by = 0.005) for (p in prange) pwr <- c(pwr, power.2(expr = myod1, constraint = list (p = p), d = 0.3, q = 1, m = 1702)$out) plot(prange, pwr, type = "l", lty = 1, xlim = c(0, 1), ylim = c(0.1, 0.9), xlab = "Porportion groups in treatment: p", ylab = "Power", main = "", col = "black") abline(v = 0.33, lty = 2, col = "Blue") par(figure)
mymdes <- power.2(expr = myod1, q = 1, power = 0.80, m = 1702) # above experssion takes parameters and outputs from od.2 function. Equivalently, each parameter can be explicitly specified. # mym <- power.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, # n = 9, p = 0.33, d = 0.3, q = 1, power = 0.8) # mymdes$out # d = 0.30
figure <- par(mfrow = c (1, 2)) MDES <- NULL nrange <- c(2:50) for (n in nrange) MDES <- c(MDES, power.2(expr = myod1, constraint = list (n = n), power = 0.8, q = 1, m = 1702)$out) plot(nrange, MDES, type = "l", lty = 1, xlim = c(0, 50), ylim = c(0.3, 0.8), xlab = "Level-1 sample size: n", ylab = "MDES", main = "", col = "black") abline(v = 9, lty = 2, col = "Blue") MDES <- NULL prange <- seq(0.05, 0.95, by = 0.005) for (p in prange) MDES <- c(MDES, power.2(expr = myod1, constraint = list (p = p), power = 0.8, q = 1, m = 1702)$out) plot(prange, MDES, type = "l", lty = 1, xlim = c(0, 1), ylim = c(0.3, 0.8), xlab = "Porportion groups in treatment: p", ylab = "MDES", main = "", col = "black") abline(v = 0.33, lty = 2, col = "Blue") par(figure)
# Required level-2 sample size calculation myJ <- power.2(cost.model = FALSE, expr = myod1, d = 0.3, q = 1, power = 0.8) # above experssion takes parameters and outputs from od.2 function. Equivalently, each parameter can be explicitly specified. # myJ <- power.2(icc = 0.2, r12 = 0.5, r22 = 0.5, # cost.model = FALSE, n = 9, p = 0.33, d = 0.3, q = 1, power = 0.8) myJ$out # J = 59 # Power calculation mypower1 <- power.2(cost.model = FALSE, expr = myod1, J = 59, d = 0.3, q = 1) mypower1$out # power = 0.80 # Minimum detectable effect size calculation mymdes1 <- power.2(cost.model = FALSE, expr = myod1, J = 59, power = 0.8, q = 1) mymdes1$out # d = 0.30
figure <- par(mfrow = c (1, 2)) pwr <- NULL mrange <- c(300:3000) for (m in mrange) pwr <- c(pwr, power.2(expr = myod1, d = 0.3, q = 1, m = m)$out) plot(mrange, pwr, type = "l", lty = 1, xlim = c(300, 3000), ylim = c(0, 1), xlab = "Budget", ylab = "Power", main = "", col = "black") abline(v = 1702, lty = 2, col = "Blue") pwr <- NULL Jrange <- c(4:100) for (J in Jrange) pwr <- c(pwr, power.2(expr = myod1, cost.model = FALSE, d = 0.3, q = 1, J = J)$out) plot(Jrange, pwr, type = "l", lty = 1, xlim = c(4, 100), ylim = c(0, 1), xlab = "Level-2 sample size: J", ylab = "Power", main = "", col = "black") abline(v = 59, lty = 2, col = "Blue") par(figure)
Please see examples in corresponding functions by uncommenting below lines.
# ?power.1 # ?power.3 # ?power.4 # ?power.2m # ?power.3m # ?power.4m
Calculate the relative efficiency (RE) of two designs, this function uses the returns from od function
Based on above examples in od functions, calculate the relative efficiency
# relative efficiency (RE) of a constrained design comparing with the optimal design myre <- re(od = myod1, subod= myod2) myre$re # get the output (i.e., RE = 0.88) # relative efficiency (RE) of a constrained design comparing with the unconstrained optimal one myre <- re(od = myod1, subod= myod3) # relative efficiency (RE) of a constrained design comparing with the unconstrained optimal one myre <- re(od = myod1, subod= myod4)
For additional examples, please see example sections in corresponding od functions by uncommenting below lines.
# ?od.1 # ?od.2 # ?od.3 # ?od.4 # ?od.2m # ?od.3m # ?od.4m
Below is a simple example to calculate optimal sample allocation and perform statistical power analyses.
# Optimal sample allocation and statistical power for randomized controlled trials myod <- od.1.111(a = .3, b = .5, c1 = 10, c1t = 100, verbose = FALSE) mypower <- power.1.111(expr = myod, power = .8) # mypower # Conventional power analyses mypower <- power.1.111(cost.model = FALSE, a = .3, b = .5, test = "joint", power = .8, p =.5) # mypower
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.