puniform: p-uniform

In puniform: Meta-Analysis Methods Correcting for Publication Bias

p-uniform

Description

Function to apply the p-uniform method for one-sample mean, two-independent means, and one raw correlation coefficient as described in van Assen, van Aert, and Wicherts (2015) and van Aert, Wicherts, and van Assen (2016).

Usage

puniform(
  mi,
  ri,
  ni,
  sdi,
  m1i,
  m2i,
  n1i,
  n2i,
  sd1i,
  sd2i,
  tobs,
  yi,
  vi,
  alpha = 0.05,
  side,
  method = "P",
  plot = FALSE
)

Arguments

mi	A vector of group means for one-sample means
ri	A vector of raw correlations
ni	A vector of sample sizes for one-sample means and correlations
sdi	A vector of standard deviations for one-sample means
m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
tobs	A vector of t-values
yi	A vector of standardized effect sizes (see Details)
vi	A vector of sampling variances belonging to the standardized effect sizes (see Details)
alpha	A numerical value specifying the alpha level as used in primary studies (default is 0.05, see Details).
side	A character indicating whether the effect sizes in the primary studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
method	A character indicating the method to be used "P" (default), "LNP", "LN1MINP", "KS", "AD", or "ML"
plot	A logical indicating whether a plot showing the relation between observed and expected p-values has to be rendered (default is TRUE)

Details

Three different effect size measures can be used as input for the puniform function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Both options should be accompanied with input for the arguments side, method, and alpha. See the Example section for examples. Raw correlation coefficients can be analyzed by supplying the raw correlation coefficients ri and sample sizes and ni to the puni_star function next to input for the arguments side, method, and alpha. Note that the method internally transforms the raw correlation coefficients to Fisher's z correlation coefficients. The output of the function also shows the results for the Fisher's z correlation coefficient. Hence, the results need to be transformed to raw correlation coefficients if this is preferred by the user.

It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments. However, extensive knowledge about computing standardized effect sizes and its sampling variances is required and specifying standardized effect sizes and sampling variances is not recommended to be used if the p-values in the primary studies are not computed with a z-test. In case the p-values in the primary studies were computed with, for instance, a t-test, the p-values of a z-test and t-test do not exactly coincide and studies may be incorrectly included in the analyses. Furthermore, critical values in the primary studies cannot be transformed to critical z-values if yi and vi are used as input. This yields less accurate results.

The puniform function assumes that two-tailed hypothesis tests were conducted in the primary studies. In case one-tailed hypothesis tests were conducted in the primary studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to p-uniform.

Note that only one effect size measure can be specified at a time. A combination of effect size measures usually causes true heterogeneity among effect sizes and including different effect size measures is therefore not recommended.

Six different estimators can be used when applying p-uniform. The P method is based on the distribution of the sum of independent uniformly distributed random variables (Irwin-Hall distribution) and is the recommended estimator (van Aert et al., 2016). The ML estimator refers to effect size estimation with maximum likelihood. Profile likelihood confidence intervals are computed, and likelihood ratio tests are used for the test of no effect and publication bias test if ML is used. The LNP estimator refers to Fisher's method (1950, Chapter 4) for combining p-values and the LN1MINP estimator first computes 1 - p-value in each study before applying Fisher's method on these transformed p-values (van Assen et al., 2015). KS and AD respectively use the Kolmogorov-Smirnov test (Massey, 1951) and the Anderson-Darling test (Anderson & Darling, 1954) for testing whether the (conditional) p-values follow a uniform distribution.

Value

est	p-uniform's effect size estimate
ci.lb	lower bound of p-uniform's confidence interval
ci.ub	upper bound of p-uniform's confidence interval
ksig	number of significant studies
L.0	test statistic of p-uniform's test of null-hypothesis of no effect (for method "P" a z-value)
pval.0	one-tailed p-value of p-uniform's test of null-hypothesis of no effect
L.pb	test statistic of p-uniform's publication bias test
pval.pb	one-tailed p-value of p-uniform's publication bias test
est.fe	effect size estimate based on traditional fixed-effect meta-analysis
se.fe	standard error of effect size estimate based on traditional fixed-effect meta-analysis
zval.fe	test statistic of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis
pval.fe	one-tailed p-value of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis
ci.lb.fe	lower bound of confidence interval based on traditional fixed-effect meta-analysis
ci.ub.fe	ci.ub.fe upper bound of confidence interval based on traditional fixed-effect meta-analysis
Qstat	test statistic of the Q-test for testing the null-hypothesis of homogeneity
Qpval	one-tailed p-value of the Q-test

Author(s)

Robbie C.M. van Aert [e-mail]

References

Anderson, T. W., & Darling, D. A. (1954). A test of goodness of fit. Journal of the American Statistical Association, 49(268), 765-769.

Fisher, R. A. (1950). Statistical methods for research workers (11th ed.). London: Oliver & Boyd.

Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68-78.

Van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting meta-analyses on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713-729. doi:10.1177/1745691616650874

Van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods, 20(3), 293-309. doi: http://dx.doi.org/10.1037/met0000025

Examples

### Load data from meta-analysis by McCall and Carriger (1993)
data(data.mccall93)

### Apply p-uniform method
puniform(ri = data.mccall93$ri, ni = data.mccall93$ni, side = "right", 
method = "LNP", plot = TRUE)

### Generate example data for one-sample means design
set.seed(123)
ni <- 100
sdi <- 1
mi <- rnorm(8, mean = 0.2, sd = sdi/sqrt(ni))
tobs <- mi/(sdi/sqrt(ni))

### Apply p-uniform method based on sample means
puniform(mi = mi, ni = ni, sdi = sdi, side = "right", plot = FALSE)

### Apply p-uniform method based on t-values
puniform(ni = ni, tobs = tobs, side = "right", plot = FALSE)

puniform documentation built on Sept. 19, 2023, 9:06 a.m.