meta-package: meta: Brief overview of methods and general hints

In meta: General Package for Meta-Analysis

meta: Brief overview of methods and general hints

Description

R package meta is a user-friendly general package providing standard methods for meta-analysis and supporting Schwarzer et al. (2015), https://link.springer.com/book/10.1007/978-3-319-21416-0.

Details

R package meta (Schwarzer, 2007; Balduzzi et al., 2019) provides the following statistical methods for meta-analysis.

1. Common effect (also called fixed effect) and random effects model:

   • Meta-analysis of continuous outcome data (metacont)

   • Meta-analysis of binary outcome data (metabin)

   • Meta-analysis of incidence rates (metainc)

   • Generic inverse variance meta-analysis (metagen)

   • Meta-analysis of single correlations (metacor)

   • Meta-analysis of single means (metamean)

   • Meta-analysis of single proportions (metaprop)

   • Meta-analysis of single incidence rates (metarate)

2. Several plots for meta-analysis:

   • Forest plot (forest.meta, forest.metabind)

   • Funnel plot (funnel.meta)

   • Galbraith plot / radial plot (radial.meta)

   • L'Abbe plot for meta-analysis with binary outcome data (labbe.metabin, labbe.default)

   • Baujat plot to explore heterogeneity in meta-analysis (baujat.meta)

   • Bubble plot to display the result of a meta-regression (bubble.metareg)

3. Three-level meta-analysis model (Van den Noortgate et al., 2013)

4. Generalised linear mixed models (GLMMs) for binary and count data (Stijnen et al., 2010) (metabin, metainc, metaprop, and metarate)

5. Various estimators for the between-study variance \tau^2 in a random effects model (Veroniki et al., 2016); see description of argument method.tau below

6. Hartung-Knapp method for random effects meta-analysis (Hartung & Knapp, 2001a,b), see description of arguments method.random.ci and adhoc.hakn.ci below

7. Kenward-Roger method for random effects meta-analysis (Partlett and Riley, 2017), see description of arguments method.random.ci and method.predict below

8. Prediction interval for the treatment effect of a new study (Higgins et al., 2009; Partlett and Riley, 2017; Nagashima et al., 2019), see description of argument method.predict below

9. Statistical tests for funnel plot asymmetry (metabias.meta, metabias.rm5) and trim-and-fill method (trimfill.meta, trimfill.default) to evaluate bias in meta-analysis

10. Meta-regression (metareg)

11. Cumulative meta-analysis (metacum) and leave-one-out meta-analysis (metainf)

12. Import data from Review Manager 5 (read.rm5), see also metacr to conduct meta-analysis for a single comparison and outcome from a Cochrane review

Additional statistical meta-analysis methods are provided by add-on R packages:

• Frequentist methods for network meta-analysis (R package netmeta)

• Statistical methods for sensitivity analysis in meta-analysis (R package metasens)

• Statistical methods for meta-analysis of diagnostic accuracy studies with several cutpoints (R package diagmeta)

In the following, more details on available and default statistical meta-analysis methods are provided and R function settings.meta is briefly described which can be used to change the default settings. Additional information on meta-analysis objects and available summary measures can be found on the help pages meta-object and meta-sm.

Estimation of between-study variance

The following methods are available in all meta-analysis functions to estimate the between-study variance \tau^2.

Argument	Method
method.tau = "REML"	Restricted maximum-likelihood estimator (Viechtbauer, 2005)
	(default)
method.tau = "PM"	Paule-Mandel estimator (Paule and Mandel, 1982)
method.tau = "DL"	DerSimonian-Laird estimator (DerSimonian and Laird, 1986)
method.tau = "ML"	Maximum-likelihood estimator (Viechtbauer, 2005)
method.tau = "HS"	Hunter-Schmidt estimator (Hunter and Schmidt, 2015)
method.tau = "SJ"	Sidik-Jonkman estimator (Sidik and Jonkman, 2005)
method.tau = "HE"	Hedges estimator (Hedges and Olkin, 1985)
method.tau = "EB"	Empirical Bayes estimator (Morris, 1983)

For GLMMs, only the maximum-likelihood method is available.

Historically, the DerSimonian-Laird method was the de facto standard to estimate the between-study variance \tau^2 and is the default in some software packages including Review Manager 5 (RevMan 5) and R package meta, version 4 and below. However, its role has been challenged and especially the REML and Paule-Mandel estimators have been recommended (Veroniki et al., 2016; Langan et al., 2019). Accordingly, the currenct default in R package meta is the REML estimator.

The following R command could be used to employ the Paule-Mandel instead of the REML estimator in all meta-analyses of the current R session:

• settings.meta(method.tau = "PM")

Other estimators for \tau^2 could be selected in a similar way.

Note, for binary outcomes, two variants of the DerSimonian-Laird estimator are available if the Mantel-Haenszel method is used for pooling. If argument Q.Cochrane = TRUE (default), the heterogeneity statistic Q is based on the Mantel-Haenszel instead of the inverse variance estimator under the common effect model. This is the estimator for \tau^2 implemented in RevMan 5.

Confidence interval for random effects estimate

The following methods are available in all meta-analysis functions to calculate a confidence interval for the random effects estimate.

Argument	Method
method.random.ci = "classic"	Based on standard normal quantile
	(DerSimonian and Laird, 1986) (default)
method.random.ci = "HK"	Method by Hartung and Knapp (2001a/b)
method.random.ci = "KR"	Kenward-Roger method (Partlett and Riley, 2017)

DerSimonian and Laird (1986) introduced the classic random effects model using a quantile of the standard normal distribution to calculate a confidence interval for the random effects estimate. This method implicitly assumes that the weights in the random effects meta-analysis are not estimated but given. Particularly, the uncertainty in the estimation of the between-study variance \tau^2 is ignored.

Hartung and Knapp (2001a,b) proposed an alternative method for random effects meta-analysis based on a refined variance estimator for the treatment estimate and a quantile of a t-distribution with k-1 degrees of freedom where k corresponds to the number of studies in the meta-analysis.

The Kenward-Roger method is only available for the REML estimator (method.tau = "REML") of the between-study variance \tau^2 (Partlett and Riley, 2017). This method is based on an adjusted variance estimate for the random effects estimate. Furthermore, a quantile of a t-distribution with adequately modified degrees of freedom is used to calculate the confidence interval.

For GLMMs and three-level models, the Kenward-Roger method is not available, but a method similar to Knapp and Hartung (2003) is used if method.random.ci = "HK". For this method, the variance estimator is not modified, however, a quantile of a t-distribution with k-1 degrees of freedom is used; see description of argument test in rma.glmm and rma.mv.

Simulation studies (Hartung and Knapp, 2001a,b; IntHout et al., 2014; Langan et al., 2019) show improved coverage probabilities of the Hartung-Knapp method compared to the classic random effects method. However, in rare settings with very homogeneous treatment estimates, the Hartung-Knapp variance estimate can be arbitrarily small resulting in a very narrow confidence interval (Knapp and Hartung, 2003; Wiksten et al., 2016). In such cases, an ad hoc variance correction has been proposed by utilising the variance estimate from the classic random effects model with the Hartung-Knapp method (Knapp and Hartung, 2003; IQWiQ, 2022). An alternative ad hoc approach is to use the confidence interval of the classic common or random effects meta-analysis if it is wider than the interval from the Hartung-Knapp method (Wiksten et al., 2016; Jackson et al., 2017).

Argument adhoc.hakn.ci can be used to choose the ad hoc correction for the Hartung-Knapp (HK) method:

Argument	Ad hoc method
adhoc.hakn.ci = ""	no ad hoc correction (default)
adhoc.hakn.ci = "se"	use variance correction if HK standard error is smaller
	than standard error from classic random effects
	meta-analysis (Knapp and Hartung, 2003)
adhoc.hakn.ci = "IQWiG6"	use variance correction if HK confidence interval
	is narrower than CI from classic random effects model
	with DerSimonian-Laird estimator (IQWiG, 2022)
adhoc.hakn.ci = "ci"	use wider confidence interval of classic random effects
	and HK meta-analysis
	(Hybrid method 2 in Jackson et al., 2017)

For GLMMs and three-level models, the ad hoc variance corrections are not available.

Prediction interval

The following methods are available in all meta-analysis functions to calculate a prediction interval for the treatment effect in a single new study.

Argument	Method
method.predict = "HTS"	Based on t-distribution (Higgins et al., 2009) (default)
method.predict = "HK"	Hartung-Knapp method (Partlett and Riley, 2017)
method.predict = "KR"	Kenward-Roger method (Partlett and Riley, 2017)
method.predict = "NNF"	Bootstrap approach (Nagashima et al., 2019)
method.predict = "S"	Based on standard normal quantile (Skipka, 2006)

By default (method.predict = "HTS"), the prediction interval is based on a t-distribution with k-2 degrees of freedom where k corresponds to the number of studies in the meta-analysis, see equation (12) in Higgins et al. (2009).

The Kenward-Roger method is only available for the REML estimator (method.tau = "REML") of the between-study variance \tau^2 (Partlett and Riley, 2017). This method is based on an adjusted variance estimate for the random effects estimate. Furthermore, a quantile of a t-distribution with adequately modified degrees of freedom minus 1 is used to calculate the prediction interval.

The bootstrap approach is only available if R package pimeta is installed (Nagashima et al., 2019). Internally, the pima function is called with argument method = "boot". Argument seed.predict can be used to get a reproducible bootstrap prediction interval and argument seed.predict.subgroup for reproducible bootstrap prediction intervals in subgroups.

The method of Skipka (2006) ignores the uncertainty in the estimation of the between-study variance \tau^2 and thus has too narrow limits for meta-analyses with a small number of studies.

For GLMMs and three-level models, the Kenward-Roger method and the bootstrap approach are not available, but a method similar to Knapp and Hartung (2003) is used if method.random.ci = "HK". For this method, the variance estimator is not modified, however, a quantile of a t-distribution with k-2 degrees of freedom is used; see description of argument test in rma.glmm and rma.mv.

Argument adhoc.hakn.pi can be used to choose the ad hoc correction for the Hartung-Knapp method:

Argument	Ad hoc method
adhoc.hakn.pi = ""	no ad hoc correction (default)
adhoc.hakn.pi = "se"	use variance correction if HK standard error is smaller

For GLMMs and three-level models, the ad hoc variance corrections are not available.

Confidence interval for the between-study variance

The following methods are available in all meta-analysis functions to calculate a confidence interval for \tau^2 and \tau.

Argument	Method
method.tau.ci = "J"	Method by Jackson (2013)
method.tau.ci = "BJ"	Method by Biggerstaff and Jackson (2008)
method.tau.ci = "QP"	Q-Profile method (Viechtbauer, 2007)
method.tau.ci = "PL"	Profile-Likelihood method for three-level
	meta-analysis model (Van den Noortgate et al., 2013)
method.tau.ci = ""	No confidence interval

The first three methods have been recommended by Veroniki et al. (2016). By default, the Jackson method is used for the DerSimonian-Laird estimator of \tau^2 and the Q-profile method for all other estimators of \tau^2.

The Profile-Likelihood method is the only method available for the three-level meta-analysis model.

For GLMMs, no confidence intervals for \tau^2 and \tau are calculated.

Change default settings for R session

R function settings.meta can be used to change the previously described and several other default settings for the current R session.

Some pre-defined general settings are available:

• settings.meta("RevMan5")

• settings.meta("JAMA")

• settings.meta("IQWiG5")

• settings.meta("IQWiG6")

• settings.meta("geneexpr")

The first command can be used to reproduce meta-analyses from Cochrane reviews conducted with Review Manager 5 (RevMan 5, https://training.cochrane.org/online-learning/core-software/revman) and specifies to use a RevMan 5 layout in forest plots.

The second command can be used to generate forest plots following instructions for authors of the Journal of the American Medical Association (https://jamanetwork.com/journals/jama/pages/instructions-for-authors/). Study labels according to JAMA guidelines can be generated using labels.meta.

The next two commands implement the recommendations of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany accordinging to General Methods 5 and 6, respectively (https://www.iqwig.de/en/about-us/methods/methods-paper/).

The last setting can be used to print p-values in scientific notation and to suppress the calculation of confidence intervals for the between-study variance.

See settings.meta for more details on these pre-defined general settings.

In addition, settings.meta can be used to define individual settings for the current R session. For example, the following R command specifies the use of Hartung-Knapp and Paule-Mandel method, and the printing of prediction intervals for any meta-analysis generated after execution of this command:

• settings.meta(method.random.ci = "HK", method.tau = "PM", prediction = TRUE)

Note

Balduzzi et al. (2019) is the preferred citation in publications for meta. Type citation("meta") for a BibTeX entry of this publication.

Type help(package = "meta") for a listing of all R functions and datasets available in meta. For example, results of several meta-analyses can be combined with metabind which is useful to generate a forest plot with results of several subgroup analyses.

R package meta imports R functions from metafor (Viechtbauer, 2010) to

• estimate the between-study variance \tau^2,

• conduct meta-regression,

• estimate three-level models,

• estimate generalised linear mixed models.

To report problems and bugs

• type bug.report(package = "meta") if you do not use RStudio,

• send an email to Guido Schwarzer [e-mail] if you use RStudio.

The development version of meta is available on GitHub https://github.com/guido-s/meta/.

Author(s)

Guido Schwarzer [e-mail]

References

Balduzzi S, Rücker G, Schwarzer G (2019): How to perform a meta-analysis with R: a practical tutorial. Evidence-Based Mental Health, 22, 153–160

Biggerstaff BJ, Jackson D (2008): The exact distribution of Cochran’s heterogeneity statistic in one-way random effects meta-analysis. Statistics in Medicine, 27, 6093–110

DerSimonian R & Laird N (1986): Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–88

Hartung J, Knapp G (2001a): On tests of the overall treatment effect in meta-analysis with normally distributed responses. Statistics in Medicine, 20, 1771–82

Hartung J, Knapp G (2001b): A refined method for the meta-analysis of controlled clinical trials with binary outcome. Statistics in Medicine, 20, 3875–89

Hedges LV & Olkin I (1985): Statistical methods for meta-analysis. San Diego, CA: Academic Press

Higgins JPT, Thompson SG, Spiegelhalter DJ (2009): A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A, 172, 137–59

Hunter JE & Schmidt FL (2015): Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (Third edition). Thousand Oaks, CA: Sage

IntHout J, Ioannidis JPA, Borm GF (2014): The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology, 14, 25

IQWiG (2022): General Methods: Version 6.1. https://www.iqwig.de/en/about-us/methods/methods-paper/

Jackson D (2013): Confidence intervals for the between-study variance in random effects meta-analysis using generalised Cochran heterogeneity statistics. Research Synthesis Methods, 4, 220–229

Jackson D, Law M, Rücker G, Schwarzer G (2017): The Hartung-Knapp modification for random-effects meta-analysis: A useful refinement but are there any residual concerns? Statistics in Medicine, 36, 3923–34

Knapp G & Hartung J (2003): Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22, 2693–710

Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, et al. (2019): A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Research Synthesis Methods, 10, 83–98

Schwarzer G (2007): meta: An R package for meta-analysis. R News, 7, 40–5

Schwarzer G, Carpenter JR and Rücker G (2015): Meta-Analysis with R (Use-R!). Springer International Publishing, Switzerland

Skipka G (2006): The inclusion of the estimated inter-study variation into forest plots for random effects meta-analysis - a suggestion for a graphical representation [abstract]. XIV Cochrane Colloquium, Dublin, 23-26.

Stijnen T, Hamza TH, Ozdemir P (2010): Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Statistics in Medicine, 29, 3046–67

Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, Knapp G, et al. (2016): Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods, 7, 55–79

Van den Noortgate W, López-López JA, Marín-Martínez F, Sánchez-Meca J (2013): Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45, 576–94

Viechtbauer W (2005): Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30, 261–93

Viechtbauer W (2007): Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine, 26, 37–52

Viechtbauer W (2010): Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36, 1–48

Wiksten A, Rücker G, Schwarzer G (2016): Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Statistics in Medicine, 35, 2503–15

See Also

meta-object, meta-sm

meta documentation built on June 7, 2023, 5:08 p.m.