hybrid: hybrid

In RobbievanAert/puniform: Meta-Analysis Methods Correcting for Publication Bias

hybrid

Description

Function to statistically combine conventional and preregistered/replications studies by means of the hybrid methods as described in van Aert and van Assen (2018) and van Aert (2023).

Usage

hybrid(
  m1i,
  m2i,
  mi,
  ri,
  sd1i,
  sd2i,
  sdi,
  n1i,
  n2i,
  ni,
  tobs,
  yi,
  vi,
  conventional,
  mods = NULL,
  alpha = 0.05,
  side,
  control
)

Arguments

m1i	A vector of means in group 1 for two-independent means
m2i	A vector of means in group 2 for two-independent means
mi	A vector of group means for one-sample means
ri	A vector of raw correlations
sd1i	A vector of standard deviations in group 1 for two-independent means
sd2i	A vector of standard deviations in group 2 for two-independent means
sdi	A vector of standard deviations for one-sample means
n1i	A vector of sample sizes in group 1 for two-independent means
n2i	A vector of sample sizes in group 2 for two-independent means
ni	A vector of sample sizes for one-sample means and correlations
tobs	A vector of t-values
yi	A vector of standardized effect sizes (see Details)
vi	A vector of sampling variances belonging to the standardized effect sizes (see Details)
conventional	A vector indicating whether a study is a conventional study (indicated with a 1) and therefore was susceptible to bias. Studies not susceptible to bias are indicated with a 0
mods	A one-sided formula to specify the moderators to include. For example x1 can be included as moderator by specifying mods = ~ x1
alpha	A numerical value specifying the alpha level as used in the conventional study (default is 0.05, see Details)
side	A character indicating whether the observed effect size of the conventional studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")
control	An optional list of elements that give the user more control over the estimation procedures (see Note)

Details

Three different effect sizes can be used as input for the hybrid function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Pearson correlation coefficients can be analyzed by supplying ri and ni to the hybrid function. These correlation coefficients are internally transformed to Fisher's z correlations before analyzing the data. The results in the output are of the Fisher's z transformed correlations. It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments.

Two other arguments that need to be specified are side and conventional. side indicates whether the effect size in the conventional study was expected to be in the right-tail (side = "right" or in the left-tail (side = "left") of the distribution. The argument conventional has to be used to indicate which studies are conventional studies and expected to be susceptible to bias. A 1 indicates that a study was susceptible to bias and a 0 indicates that a study was not susceptible to bias.

It is assumed that two-tailed hypothesis tests were conducted in the conventional studies. In case one-tailed hypothesis tests were conducted in the conventional studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be supplied to the hybrid function.

Previous version

The usage of a previous version of the hybrid function was more restricted. Users could only apply the method to a single conventional study and replication. Before the addition of the extra functionality to also analyze multiple conventional studies and replications, data of the conventional study and replication were specified in vectors containing two elements with the first element being the data of the conventional study and the second one the data of the replication. In order to maintain backwards compatibility, it is still possible to analyze data like this by using the arguments m1i, m2i, mi, ri, sd1i, sd2i, sdi, n1i, n2i, ni, tobs. However, using the hybrid function in this way is now deprecated.

Value

k	total number of effect sizes
k.conventional	number of effect sizes of conventional studies
est	parameter estimates of the fixed effects of the hybrid method
tau2	estimate of the between-study variance in true effect size of hybrid method
se	standard error of the fixed effects and of the estimated between-study variance
ci.lb	lower bound of hybrid method's confidence interval of the average effect size
ci.ub	upper bound of hybrid method's confidence interval of the average effect size
L.0	test statistic of hybrid method's test of null-hypothesis of no effect. This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.0	p-value of hybrid method's test of null-hypothesis of no effect
tau2	estimate of the between-study variance in true effect size of hybrid method
tau2.lb	lower bound of hybrid method's confidence interval of the between-study variance
tau2.ub	upper bound of hybrid method's confidence interval of the between-study variance
L.het	test statistic of hybrid method's test of null-hypothesis of no heterogeneity This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test
pval.het	p-value of hybrid method's test of null-hypothesis of no heterogeneity
optim.info	model fitting results if the implementation of van Aert (2023) is used

The elements below are only returned if the deprecated implementation of van Aert and van Assen (2018) is used:

est.hyr	effect size estimate of hybridR method
ci.lb.hyr	lower bound of hybridR method's confidence interval of the effect size
ci.ub.hyr	upper bound of hybridR method's confidence interval of the effect size
L.0.hyr	test statistic of hybridR method's test of null-hypothesis of no effect
pval.0.hyr	p-value of hybridR method's test of null-hypothesis of no effect
pval.o	two-tailed p-value of conventional study
est.hy0	effect size estimate of hybrid0 method
ci.lb.hy0	lower bound of hybrid0 method's confidence interval of the effect size
ci.ub.hy0	upper bound of hybrid0 method's confidence interval of the effect size
L.0.hy0	test statistic of hybrid0 method's test of null-hypothesis of no effect
pval.0.hy0	two-tailed p-value of hybrid0 method's test of null-hypothesis of no effect
est.fe	effect size estimate based on fixed-effect meta-analysis
se.fe	standard error of effect size estimate based on traditional fixed-effect meta-analysis
zval.fe	test statistic of the null-hypothesis of no effect based on fixed-effect meta-analysis
pval.fe	two-tailed p-value of the null-hypothesis of no effect based on fixed-effect meta-analysis
ci.lb.fe	lower bound of confidence interval based on traditional fixed-effect meta-analysis
ci.ub.fe	upper bound of confidence interval based on fixed-effect meta-analysis
est.repl	effect size estimate of replication
se.repl	standard error of replication's effect size estimate
ci.lb.repl	lower bound of replication's confidence interval
ci.ub.repl	upper bound of replication's confidence interval
stat.repl	test statistic of replication for testing null-hypothesis of no effect
pval.repl	two-tailed p-value of replication for testing null-hypothesis of no effect

Note

The control argument in the hybrid function is an optional argument that gives the user more control over the estimation procedures. This can be especially useful if estimation of the method does not converge and NAs are returned by the function. The control argument should be specified as a list containing one or more elements. For example, control = list(verbose = TRUE) Default values are used if an element is not specified. The following elements can be specified by the user:

• int: A vector of length two that indicates the lower and upper bound of the interval that is used for estimating the effect size. The effect size estimate should be included in this interval. Its default values are -10 for the first element and the maximum effect size of a study included in the analysis + 1 as the second element. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

• est.ci: A vector of length two indicating the values that are subtracted from and added to the estimate of the effect size for computing the 95% confidence intervals. Its default values are (50, 1). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (est-50, est) and (est, est+1), respectively.

• tau2.ci A vector of length two indicating the values that are subtracted from and added to the estimate of the between-study variance for computing the 95% confidence intervals. Its default values are (0.5, 0.5). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (tau2-0.5, tau2) and (tau2, tau2+0.5), respectively.

• tol: A number indicating the desired accuracy of the estimates. Its default value is .Machine$double.eps^0.25. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

• verbose: A logical indicating whether information should be printed about the estimation procedure. Its default value is FALSE.

• par: Starting values for the optimization procedure in case of the implementation of van Aert (2023). The default values are zeros.

• implementation: A character indicating whether the implementation of van Aert and van Assen (2018) based on a single conventional and single replication should be used (implementation = "two") or whether the implementation of van Aert (2023) should be used that allows for including more than two studies (implementation = "multiple").

• optimizer: A character indicating the optimizer that is used if the implementation of van Aert (2023) is used. The default value is "Nelder-Mead". The optim function is used for optimization, so the optimization methods implemented in the optim function can be used. See the documentation of the optim function for more information.

• type: A character vector indicating whether Wald-based hypothesis tests and confidence intervals are preferred (type = "Wald") or likelihood-ratio tests and profile likelihood confidence intervals (type = "profile"). The default is "Wald/profile" which implies that Wald tests and confidence intervals are computed for the fixed effects and likelihood-ratio tests and profile likelihood confidence intervals for the between-study variance.

Author(s)

Robbie C.M. van Aert [e-mail]

References

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for statistically combining a biased original study and replication. Behavior Research Methods, 50(4): 1515-1539. doi:10.3758/s13428-017-0967-6

van Aert, R.C.M. (2023). Empowering meta-analysis by taking advantage of preregistered studies: The extended hybrid meta-analysis method. Manuscript in preparation.

Examples

### Application using the implementation of van Aert (2023) with more than two studies.
# Note that these data come from the "metadat" R package that needs to be loaded.
# The data come from the meta-analyses by Lehmann et al. (2018). See the help 
# file of dat.lehmann2018 for more information about this meta-analysis.
library(metadat)
dat <- dat.lehmann2018

### Create a new object to indicate which studies were conventional studies and 
# which ones were preregistered
dat$conventional <- ifelse(dat$Preregistered == "Not Pre-Registered", 1, 0)

### Lehmann et al. split the analyses for males and females. We only use the data
# of females for this example.
red_romance_femalep <- dat[dat$Gender == "Females", ]

### Prepare data for the analysis
yi <- red_romance_femalep$yi
vi <- red_romance_femalep$vi
conventional <- red_romance_femalep$conventional

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right")

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", 
control = list(type = "profile"))

### Include Color_Match as moderator
Color_Match <- red_romance_femalep$Color_Match

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match)

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match, 
control = list(type = "profile"))

### Application using the implementation of van Aert and van Assen (2018). 
# The hybrid method is applied to example on page 5 of van Aert and van Assen 
# (2018).

pval <- c(0.03, 0.3) # p-value conventional study and replication
n1i <- n2i <- c(40, 80) # Sample sizes per group 
tobs <- qt(pval/2, df = n1i+n2i-2, lower.tail = FALSE) # Observed t-values 

### Apply hybrid method using the implementation of van Aert and van Assen (2018)
hybrid(tobs = tobs, n1i = n1i, n2i = n2i, side = "right", conventional = c(1,0), 
control = list(implementation = "two"))
 

RobbievanAert/puniform documentation built on Sept. 22, 2023, 2:53 a.m.