rpLM: Calculates the 'replication probability of significance' of...

In reverseR: Linear Regression Stability to Significance Reversal

Calculates the 'replication probability of significance' of an 'lm' object

Description

This function uses a bootstrap approach to calculate the replication probability of significance, which answers the question "if we repeat this linear regression under identical conditions (similar sample size, similar residual variance), what is the probability of observing significance (or non-significance) similar to the original data?".

Usage

rpLM(model, alpha = 0.05, R = 10000, plot = TRUE, verbose = TRUE, ...) 

Arguments

model	a linear model of class lm.
alpha	the \alpha-level to use as the threshold border.
R	the number of bootstrap resamples, see bootLM.
plot	logical. If TRUE, a stripchart of the bootstrap P-values, the original P-value and the \alpha-level is displayed.
verbose	logical. If TRUE, the analysis steps are written to the console.
...	other parameters to be supplied to bootLM.

Details

The approach here is along the lines of Boos & Stefanski (2011), which investigated the replication probability of the P-value, as opposed to the works of Goodman (1992), Shao & Chow (2002) and Miller (2009), where the effect size is used. In our context, for a given linear model and using a bootstrap approach, the replication probability is the proportion of bootstrap P-values with \tilde{P} \leq \alpha when the original model is significant, or \tilde{P} > \alpha when not. Hence, we employ the bootstrap to assess the sampling variability of the P-value, not the sampling variability of the P-value under H_0, as is common, thereby preserving the non-null property of the data.

Bootstrap results are obtained from non-parametric cases bootstrapping ("np.cases"), non-parametric residuals bootstrapping ("np.resid") and parametric residuals bootstrapping ("p.resid"), see bootLM.

Value

A vector with the three different bootstrap results as described above.

Author(s)

Andrej-Nikolai Spiess

References

Ecological Models and Data in R.
Chapter 5: Stochastic simulation and power analysis.
Benjamin M. Bolker.
Princeton University Press (2008).

P-Value Precision and Reproducibility.
Boos DD & Stefanski LA.
Am Stat, 65, 2011, 213-212.

A comment on replication, p-values and evidence.
Goodman SN.
Stat Med, 11, 1992, 875-879.

Reproducibility probability in clinical trials.
Shao J & Chow SC.
Stat Med, 21, 2002, 1727-1742.

What is the probability of replicating a statistically significant effect?
Miller J.
Psych Bull & Review, 16, 2009, 617-640.

Examples

set.seed(125)
a <- 1:20
b <- 5 + 0.08 * a + rnorm(length(a), 0, 1)
LM1 <- lm(b ~ a)
summary(LM1)
rpLM(LM1, R = 100)

reverseR documentation built on Sept. 12, 2024, 7:32 a.m.