hybrid: hybrid

View source: R/hybrid.R

hybridR Documentation

hybrid

Description

Function to statistically combine conventional and preregistered/replications studies by means of the hybrid methods as described in van Aert and van Assen (2018) and van Aert (2023).

Usage

hybrid(
  m1i,
  m2i,
  mi,
  ri,
  sd1i,
  sd2i,
  sdi,
  n1i,
  n2i,
  ni,
  tobs,
  yi,
  vi,
  conventional,
  mods = NULL,
  alpha = 0.05,
  side,
  control
)

Arguments

m1i

A vector of means in group 1 for two-independent means

m2i

A vector of means in group 2 for two-independent means

mi

A vector of group means for one-sample means

ri

A vector of raw correlations

sd1i

A vector of standard deviations in group 1 for two-independent means

sd2i

A vector of standard deviations in group 2 for two-independent means

sdi

A vector of standard deviations for one-sample means

n1i

A vector of sample sizes in group 1 for two-independent means

n2i

A vector of sample sizes in group 2 for two-independent means

ni

A vector of sample sizes for one-sample means and correlations

tobs

A vector of t-values

yi

A vector of standardized effect sizes (see Details)

vi

A vector of sampling variances belonging to the standardized effect sizes (see Details)

conventional

A vector indicating whether a study is a conventional study (indicated with a 1) and therefore was susceptible to bias. Studies not susceptible to bias are indicated with a 0

mods

A one-sided formula to specify the moderators to include. For example x1 can be included as moderator by specifying mods = ~ x1

alpha

A numerical value specifying the alpha level as used in the conventional study (default is 0.05, see Details)

side

A character indicating whether the observed effect size of the conventional studies are in the right-tail of the distribution (i.e., positive) or in the left-tail of the distribution (i.e., negative) (either "right" or "left")

control

An optional list of elements that give the user more control over the estimation procedures (see Note)

Details

Three different effect sizes can be used as input for the hybrid function: one-sample means, two-independent means, and raw correlation coefficients. Analyzing one-sample means and two-independent means can be done by either providing the function group means (mi or m1i and m2i), standard deviations (sdi or sd1i and sd2i), and sample sizes (ni or n1i and n2i) or t-values (tobs) and sample sizes (ni or n1i and n2i). Pearson correlation coefficients can be analyzed by supplying ri and ni to the hybrid function. These correlation coefficients are internally transformed to Fisher's z correlations before analyzing the data. The results in the output are of the Fisher's z transformed correlations. It is also possible to specify the standardized effect sizes and its sampling variances directly via the yi and vi arguments.

Two other arguments that need to be specified are side and conventional. side indicates whether the effect size in the conventional study was expected to be in the right-tail (side = "right" or in the left-tail (side = "left") of the distribution. The argument conventional has to be used to indicate which studies are conventional studies and expected to be susceptible to bias. A 1 indicates that a study was susceptible to bias and a 0 indicates that a study was not susceptible to bias.

It is assumed that two-tailed hypothesis tests were conducted in the conventional studies. In case one-tailed hypothesis tests were conducted in the conventional studies, the alpha level has to be multiplied by two. For example, if one-tailed hypothesis tests were conducted with an alpha level of .05, an alpha of 0.1 has to be supplied to the hybrid function.

Previous version

The usage of a previous version of the hybrid function was more restricted. Users could only apply the method to a single conventional study and replication. Before the addition of the extra functionality to also analyze multiple conventional studies and replications, data of the conventional study and replication were specified in vectors containing two elements with the first element being the data of the conventional study and the second one the data of the replication. In order to maintain backwards compatibility, it is still possible to analyze data like this by using the arguments m1i, m2i, mi, ri, sd1i, sd2i, sdi, n1i, n2i, ni, tobs. However, using the hybrid function in this way is now deprecated.

Value

k

total number of effect sizes

k.conventional

number of effect sizes of conventional studies

est

parameter estimates of the fixed effects of the hybrid method

tau2

estimate of the between-study variance in true effect size of hybrid method

se

standard error of the fixed effects and of the estimated between-study variance

ci.lb

lower bound of hybrid method's confidence interval of the average effect size

ci.ub

upper bound of hybrid method's confidence interval of the average effect size

L.0

test statistic of hybrid method's test of null-hypothesis of no effect. This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test

pval.0

p-value of hybrid method's test of null-hypothesis of no effect

tau2

estimate of the between-study variance in true effect size of hybrid method

tau2.lb

lower bound of hybrid method's confidence interval of the between-study variance

tau2.ub

upper bound of hybrid method's confidence interval of the between-study variance

L.het

test statistic of hybrid method's test of null-hypothesis of no heterogeneity This is either a z-value (zval in the output) or a chi-square value (LR in the output) in case of a likelihood-ratio test

pval.het

p-value of hybrid method's test of null-hypothesis of no heterogeneity

optim.info

model fitting results if the implementation of van Aert (2023) is used

The elements below are only returned if the deprecated implementation of van Aert and van Assen (2018) is used:

est.hyr

effect size estimate of hybridR method

ci.lb.hyr

lower bound of hybridR method's confidence interval of the effect size

ci.ub.hyr

upper bound of hybridR method's confidence interval of the effect size

L.0.hyr

test statistic of hybridR method's test of null-hypothesis of no effect

pval.0.hyr

p-value of hybridR method's test of null-hypothesis of no effect

pval.o

two-tailed p-value of conventional study

est.hy0

effect size estimate of hybrid0 method

ci.lb.hy0

lower bound of hybrid0 method's confidence interval of the effect size

ci.ub.hy0

upper bound of hybrid0 method's confidence interval of the effect size

L.0.hy0

test statistic of hybrid0 method's test of null-hypothesis of no effect

pval.0.hy0

two-tailed p-value of hybrid0 method's test of null-hypothesis of no effect

est.fe

effect size estimate based on fixed-effect meta-analysis

se.fe

standard error of effect size estimate based on traditional fixed-effect meta-analysis

zval.fe

test statistic of the null-hypothesis of no effect based on fixed-effect meta-analysis

pval.fe

two-tailed p-value of the null-hypothesis of no effect based on fixed-effect meta-analysis

ci.lb.fe

lower bound of confidence interval based on traditional fixed-effect meta-analysis

ci.ub.fe

upper bound of confidence interval based on fixed-effect meta-analysis

est.repl

effect size estimate of replication

se.repl

standard error of replication's effect size estimate

ci.lb.repl

lower bound of replication's confidence interval

ci.ub.repl

upper bound of replication's confidence interval

stat.repl

test statistic of replication for testing null-hypothesis of no effect

pval.repl

two-tailed p-value of replication for testing null-hypothesis of no effect

Note

The control argument in the hybrid function is an optional argument that gives the user more control over the estimation procedures. This can be especially useful if estimation of the method does not converge and NAs are returned by the function. The control argument should be specified as a list containing one or more elements. For example, control = list(verbose = TRUE) Default values are used if an element is not specified. The following elements can be specified by the user:

  • int: A vector of length two that indicates the lower and upper bound of the interval that is used for estimating the effect size. The effect size estimate should be included in this interval. Its default values are -10 for the first element and the maximum effect size of a study included in the analysis + 1 as the second element. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

  • est.ci: A vector of length two indicating the values that are subtracted from and added to the estimate of the effect size for computing the 95% confidence intervals. Its default values are (50, 1). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (est-50, est) and (est, est+1), respectively.

  • tau2.ci A vector of length two indicating the values that are subtracted from and added to the estimate of the between-study variance for computing the 95% confidence intervals. Its default values are (0.5, 0.5). To give an example, estimates for the lower and upper bound around the effect size estimate are searched on the interval (tau2-0.5, tau2) and (tau2, tau2+0.5), respectively.

  • tol: A number indicating the desired accuracy of the estimates. Its default value is .Machine$double.eps^0.25. This control argument is only applicable to the implementation of van Aert and van Assen (2018).

  • verbose: A logical indicating whether information should be printed about the estimation procedure. Its default value is FALSE.

  • par: Starting values for the optimization procedure in case of the implementation of van Aert (2023). The default values are zeros.

  • implementation: A character indicating whether the implementation of van Aert and van Assen (2018) based on a single conventional and single replication should be used (implementation = "two") or whether the implementation of van Aert (2023) should be used that allows for including more than two studies (implementation = "multiple").

  • optimizer: A character indicating the optimizer that is used if the implementation of van Aert (2023) is used. The default value is "Nelder-Mead". The optim function is used for optimization, so the optimization methods implemented in the optim function can be used. See the documentation of the optim function for more information.

  • type: A character vector indicating whether Wald-based hypothesis tests and confidence intervals are preferred (type = "Wald") or likelihood-ratio tests and profile likelihood confidence intervals (type = "profile"). The default is "Wald/profile" which implies that Wald tests and confidence intervals are computed for the fixed effects and likelihood-ratio tests and profile likelihood confidence intervals for the between-study variance.

Author(s)

Robbie C.M. van Aert R.C.M.vanAert@tilburguniversity.edu

References

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for statistically combining a biased original study and replication. Behavior Research Methods, 50(4): 1515-1539. doi:10.3758/s13428-017-0967-6

van Aert, R.C.M. (2023). Empowering meta-analysis by taking advantage of preregistered studies: The extended hybrid meta-analysis method. Manuscript in preparation.

Examples


### Application using the implementation of van Aert (2023) with more than two studies.
# Note that these data come from the "metadat" R package that needs to be loaded.
# The data come from the meta-analyses by Lehmann et al. (2018). See the help 
# file of dat.lehmann2018 for more information about this meta-analysis.
library(metadat)
dat <- dat.lehmann2018

### Create a new object to indicate which studies were conventional studies and 
# which ones were preregistered
dat$conventional <- ifelse(dat$Preregistered == "Not Pre-Registered", 1, 0)

### Lehmann et al. split the analyses for males and females. We only use the data
# of females for this example.
red_romance_femalep <- dat[dat$Gender == "Females", ]

### Prepare data for the analysis
yi <- red_romance_femalep$yi
vi <- red_romance_femalep$vi
conventional <- red_romance_femalep$conventional

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right")

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", 
control = list(type = "profile"))

### Include Color_Match as moderator
Color_Match <- red_romance_femalep$Color_Match

### Apply the hybrid method with Wald-type hypothesis tests and confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match)

### Apply the hybrid method with likelihood ratio hypothesis tests and profile 
# likelihood confidence intervals
hybrid(yi = yi, vi = vi, conventional = conventional, side = "right", mods = ~ Color_Match, 
control = list(type = "profile"))

### Application using the implementation of van Aert and van Assen (2018). 
# The hybrid method is applied to example on page 5 of van Aert and van Assen 
# (2018).

pval <- c(0.03, 0.3) # p-value conventional study and replication
n1i <- n2i <- c(40, 80) # Sample sizes per group 
tobs <- qt(pval/2, df = n1i+n2i-2, lower.tail = FALSE) # Observed t-values 

### Apply hybrid method using the implementation of van Aert and van Assen (2018)
hybrid(tobs = tobs, n1i = n1i, n2i = n2i, side = "right", conventional = c(1,0), 
control = list(implementation = "two"))
 

RobbievanAert/puniform documentation built on Sept. 22, 2023, 2:53 a.m.