tCalibrate: Calibration of p-values from t-tests

In pCalibrate: Bayesian Calibrations of p-Values

Description

Transforms one- and two-sided p-values from t-tests to sample-size adjusted lower bounds on the Bayes factor for the point null hypothesis against the alternative.

Usage

tCalibrate(p, n, type="two.sided", alternative="normal", transform="id")

Arguments

p	a vector of p-values
n	a vector of sample sizes
type	either "one.sided" or "two.sided". Defaults to "two.sided". Specifies if the t-test (and the corresponding p-value) is one-sided or two-sided.
alternative	either "simple" or "normal". Defaults to "normal". Specifies the alternative hypotheses for the mean to consider. See Details for more information.
transform	either "id", "log", "log2" or "log10". Defaults to "id". Specifies how to transform the lower bound on the Bayes factor. "id" corresponds to no transformation. "log" refers to the natural logarithm, "log2" to the logarithm to the base 2 and "log10" to the logarithm to the base 10.

Details

For one-sided t-tests (type="one.sided"), alternative="simple" considers all simple point alternatives for the mean and alternative="normal" all shifted and possibly mirrored
(if direction=less) half-normal distributions with mode at the null value.

For two-sided t-tests (type="two.sided"), alternative="simple" considers all two-point distributions on the mean which are symmetric with respect to the null value and
alternative="normal" all local normal distributions centered around the null value.

Note that for alternative="normal", the lower bound on the Bayes factor is the same for one-sided and two-sided p-values.

For type="one.sided", alternative="simple", the calibration is available in closed form and is given in Held & Ott (2018), equation (19).

For type="two.sided", alternative="simple", the calibration needs to be computed by numerical optimization and is given in Held & Ott (2018), equations (17) and (18).

For alternative="normal", the corresponding lower bound on the Bayes factor has been derived in Johnson (2005) and the calibration is proposed in Held & Ott (2016), Section 3.

If alternative="normal", tCalibrate() is a special case of FCalibrate(), i.e.
tCalibrate(p, n, alternative="normal") is equivalent to
FCalibrate(p, n, d=1, alternative="chi.squared").

One can show that if one considers the class of all alternatives which are symmetric with respect to the null value, one obtains the same lower bound as for the subclass of all symmetric two-point distributions.

Value

A matrix containing the lower bounds on the Bayes factors as entries, for all combinations of p-value and sample size. The values for the k-th sample size (k-th entry in the vector n) and the specified p-values are given in the k-th row.

References

Held, L. and Ott, M. (2018). On p-values and Bayes factors. Annual Review of Statistics and Its Application, 5, 393–419.

Held, L. and Ott, M. (2016). How the maximal evidence of P-values against point null hypotheses depends on sample size. The American Statistician, 70, 335–341.

Johnson, V. E. (2005). Bayes factors based on test statistics. Journal of the Royal Statistical Society, Series B 67:689–701.

See Also

FCalibrate

Examples

tCalibrate(p=c(0.05, 0.01, 0.005), n=10, type="one.sided", 
           alternative="simple")
# two-sided alternatives
tCalibrate(p=c(0.05, 0.01, 0.005), n=20, alternative="simple")
tCalibrate(p=c(0.05, 0.01, 0.005), n=c(10, 20, 50))

# for two-sided alternatives, plot the minimum Bayes factors 
# as a function of the p-value
par(mfrow=c(1,2), pty="s", las=1)

# plot for simple alternatives
p <- exp(seq(log(0.0001), log(0.3), by=0.01))
n <- c(10, 20, 50)
minBF <- tCalibrate(p, n, alternative = "simple")
# compare to the bound for large n
bound1 <- zCalibrate(p, alternative="simple")
matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", 
        ylab="Min. Bayes factor", log="xy", 
        xlab="two-sided t-test p-value", lty=1, lwd=2, 
        axes=FALSE, main="Simple alternative")
lines(p, bound1, col="gray", lty=2, lwd=2)
my.at1 <- c(0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3)
my.values1 <- as.character(c(format(c(0.0001,0.0003), nsmall=4, 
                                    digits=4, scientific=FALSE), 
                           c(0.001, 0.003, 0.01, 0.03, 0.1, 0.3)))
axis(1, at=my.at1, labels=my.values1)
my.values2 <- c(3000, 1000, 300, 100, 30, 10, 3, 1)
my.at2 <- 1/my.values2
my.ylegend2 <- c(paste("1/", my.values2[-length(my.values2)], sep=""), 
                 "1")
axis(2, at=my.at2, labels=my.ylegend2)
box()
legend("bottomright", legend=rev(c("n=10", "n=20", "n=50", "n large")), 
        lty=rev(c(rep(1,3), 2)), lwd=2, col=rev(c(1:3, "gray")))
       
# plot for local normal alternatives
p <- exp(seq(log(0.0001), log(0.3), by=0.01))
n <- c(10, 20, 50)
minBF <- tCalibrate(p, n)
bound2 <- zCalibrate(p)
matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", 
        ylab="Min. Bayes factor", log="xy", 
        xlab="two-sided t-test p-value", lty=1, lwd=2, 
        axes=FALSE, main="Local normal alternatives")
lines(p, bound2, col="gray", lty=2, lwd=2)
axis(1, at=my.at1, labels=my.values1)
axis(2, at=my.at2, my.ylegend2)
box()            
legend("bottomright", legend=rev(c("n=10", "n=20", "n=50", "n large")), 
        lty=rev(c(rep(1,3), 2)), lwd=2, col=rev(c(1:3, "gray")))

pCalibrate documentation built on March 20, 2020, 1:09 a.m.