PPpSceptical: Compute project power of the sceptical p-value

In ReplicationSuccess: Design and Analysis of Replication Studies

Description

The project power of the sceptical p-value is computed for a specified level of replication success, the relative variance, significance level and power for a standard significance test of the original study, and the alternative hypothesis.

Usage

PPpSceptical(level, c, alpha, power, alternative = "one.sided")

Arguments

level	A vector of levels of replication success
c	A vector of variance ratios of the original and replication effect estimates. This is usually the ratio of the sample size of the replication study to the sample size of the original study.
alpha	Significance level for a standard significance test in the original study.
power	Power to detect the assumed effect with a standard significance test in the original study.
alternative	Either "one.sided" or "two.sided". Defaults to "one.sided". If "one.sided" then the project power is computed for the sceptical p-value based on a one-sided assessment of replication success in the direction of the original effect estimate. If "two.sided" then the project power is computed for the sceptical p-value based on a two-sided assessment of replication success regardless of the direction of the original and replication effect estimate.

Value

The project power

Author(s)

Samuel Pawel

References

Held, L. (2020). The harmonic mean chi-squared test to substantiate scientific findings. Journal of the Royal Statistical Society: Series C (Applied Statistics), 69, 697-708. https://doi.org/10.1111/rssc.12410

Held, L., Micheloud, C. & Pawel, S. (2020). The assessment of replication success based on relative effect size. https://arxiv.org/abs/2009.07782

Maca, J., Gallo, P., Branson, M., and Maurer, W. (2002). Reconsidering some aspects of the two-trials paradigm. Journal of Biopharmaceutical Statistics, 12(2):107-119. https://doi.org/10.1081/bip-120006450

See Also

pSceptical, levelSceptical, T1EpSceptical

Examples

## compare project power for different levels of replication success
levels <- c("nominal" = levelSceptical(level = 0.025, type = "nominal"),
            "liberal" = levelSceptical(level = 0.025, type = "liberal"),
            "controlled" = levelSceptical(level = 0.025, type = "controlled"),
            "golden" = levelSceptical(level = 0.025, type = "golden"))
c <- seq(0.4, 5, by = 0.01)
alpha <- 0.025
power <- 0.9
pp <- sapply(X = levels, FUN = function(l) {
  PPpSceptical(level = l, c = c, alpha, power, alternative = "one.sided")
})

## compute project power of 2 trials rule
za <- qnorm(p = 1 - alpha)
mu <- za + qnorm(p = power)
pp2TR <- power*pnorm(q = za, mean = sqrt(c)*mu, lower.tail = FALSE)

matplot(x = c, y = pp*100, type = "l", lty = 1, lwd = 2, las = 1, log = "x",
        xlab = bquote(italic(c)), ylab = "Project power (%)", xlim = c(0.4, 5),
        ylim = c(0, 100))
lines(x = c, y = pp2TR*100, col = length(levels) + 1, lwd = 2)
abline(v = 1, lty = 2)
abline(h = 90, lty = 2, col = "lightgrey")
legend("bottomright", legend = c(names(levels), "2TR"), lty = 1, lwd = 2, 
       col = seq(1, length(levels) + 1))

ReplicationSuccess documentation built on Dec. 2, 2020, 3 p.m.