puniform: p-uniform

View source: R/puniform.R

puniformR Documentation

p-uniform

Description

Function to apply the p-uniform method for one-sample mean, two-independent means, and one raw correlation coefficient as described in van Assen, van Aert, and Wicherts (2015) and van Aert, Wicherts, and van Assen (2016).

Usage

puniform(
  mi,
  ri,
  ni,
  sdi,
  m1i,
  m2i,
  n1i,
  n2i,
  sd1i,
  sd2i,
  tobs,
  yi,
  vi,
  alpha = 0.05,
  side,
  method = "P",
  plot = FALSE
)

Arguments

mi

A vector of group means for one-sample means

ri

A vector of raw correlations

ni

A vector of sample sizes for one-sample means and correlations

sdi

A vector of standard deviations for one-sample means

m1i

A vector of means in group 1 for two-independent means

m2i

A vector of means in group 2 for two-independent means

n1i

A vector of sample sizes in group 1 for two-independent means

n2i

A vector of sample sizes in group 2 for two-independent means

sd1i

A vector of standard deviations in group 1 for two-independent means

sd2i

A vector of standard deviations in group 2 for two-independent means

tobs

A vector of t-values

yi

A vector of standardized effect sizes (see Details)

vi

A vector of sampling variances belonging to the standardized effect sizes (see Details)

alpha

A numerical value specifying the alpha level as used in primary studies (default is 0.05, see Details).

side

A character indicating whether the effect sizes in the primary studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")

method

A character indicating the method to be used "P" (default), "LNP", "LN1MINP", "KS", "AD", or "ML"

plot

A logical indicating whether a plot showing the relation between observed and expected p-values has to be rendered (default is TRUE)

Details

Three different effect size measures can be used as input for the puniform function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Both options should be accompanied with input for the arguments side, method, and alpha. See the Example section for examples. Raw correlation coefficients can be analyzed by supplying the raw correlation coefficients ri and sample sizes and ni to the puni_star function next to input for the arguments side, method, and alpha. Note that the method internally transforms the raw correlation coefficients to Fisher's z correlation coefficients. The output of the function also shows the results for the Fisher's z correlation coefficient. Hence, the results need to be transformed to raw correlation coefficients if this is preferred by the user.

It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments. However, extensive knowledge about computing standardized effect sizes and its sampling variances is required and specifying standardized effect sizes and sampling variances is not recommended to be used if the p-values in the primary studies are not computed with a z-test. In case the p-values in the primary studies were computed with, for instance, a t-test, the p-values of a z-test and t-test do not exactly coincide and studies may be incorrectly included in the analyses. Furthermore, critical values in the primary studies cannot be transformed to critical z-values if yi and vi are used as input. This yields less accurate results.

The puniform function assumes that two-tailed hypothesis tests were conducted in the primary studies. In case one-tailed hypothesis tests were conducted in the primary studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be submitted to p-uniform.

Note that only one effect size measure can be specified at a time. A combination of effect size measures usually causes true heterogeneity among effect sizes and including different effect size measures is therefore not recommended.

Six different estimators can be used when applying p-uniform. The P method is based on the distribution of the sum of independent uniformly distributed random variables (Irwin-Hall distribution) and is the recommended estimator (van Aert et al., 2016). The ML estimator refers to effect size estimation with maximum likelihood. Profile likelihood confidence intervals are computed, and likelihood ratio tests are used for the test of no effect and publication bias test if ML is used. The LNP estimator refers to Fisher's method (1950, Chapter 4) for combining p-values and the LN1MINP estimator first computes 1 - p-value in each study before applying Fisher's method on these transformed p-values (van Assen et al., 2015). KS and AD respectively use the Kolmogorov-Smirnov test (Massey, 1951) and the Anderson-Darling test (Anderson & Darling, 1954) for testing whether the (conditional) p-values follow a uniform distribution.

Value

est

p-uniform's effect size estimate

ci.lb

lower bound of p-uniform's confidence interval

ci.ub

upper bound of p-uniform's confidence interval

ksig

number of significant studies

L.0

test statistic of p-uniform's test of null-hypothesis of no effect (for method "P" a z-value)

pval.0

one-tailed p-value of p-uniform's test of null-hypothesis of no effect

L.pb

test statistic of p-uniform's publication bias test

pval.pb

one-tailed p-value of p-uniform's publication bias test

est.fe

effect size estimate based on traditional fixed-effect meta-analysis

se.fe

standard error of effect size estimate based on traditional fixed-effect meta-analysis

zval.fe

test statistic of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis

pval.fe

one-tailed p-value of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis

ci.lb.fe

lower bound of confidence interval based on traditional fixed-effect meta-analysis

ci.ub.fe

ci.ub.fe upper bound of confidence interval based on traditional fixed-effect meta-analysis

Qstat

test statistic of the Q-test for testing the null-hypothesis of homogeneity

Qpval

one-tailed p-value of the Q-test

Author(s)

Robbie C.M. van Aert R.C.M.vanAert@tilburguniversity.edu

References

Anderson, T. W., & Darling, D. A. (1954). A test of goodness of fit. Journal of the American Statistical Association, 49(268), 765-769.

Fisher, R. A. (1950). Statistical methods for research workers (11th ed.). London: Oliver & Boyd.

Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68-78.

Van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting meta-analyses on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713-729. doi:10.1177/1745691616650874

Van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods, 20(3), 293-309. doi: http://dx.doi.org/10.1037/met0000025

Examples

### Load data from meta-analysis by McCall and Carriger (1993)
data(data.mccall93)

### Apply p-uniform method
puniform(ri = data.mccall93$ri, ni = data.mccall93$ni, side = "right", 
method = "LNP", plot = TRUE)

### Generate example data for one-sample means design
set.seed(123)
ni <- 100
sdi <- 1
mi <- rnorm(8, mean = 0.2, sd = sdi/sqrt(ni))
tobs <- mi/(sdi/sqrt(ni))

### Apply p-uniform method based on sample means
puniform(mi = mi, ni = ni, sdi = sdi, side = "right", plot = FALSE)

### Apply p-uniform method based on t-values
puniform(ni = ni, tobs = tobs, side = "right", plot = FALSE)


RobbievanAert/puniform documentation built on Sept. 22, 2023, 2:53 a.m.