fat: Regression tests for detecting funnel plot asymmetry

In metamisc: Meta-Analysis of Diagnosis and Prognosis Research Studies

Description

The presence of small-study effects is a common threat to systematic reviews and meta-analyses, especially when it is due to publication bias, which occurs when small primary studies are more likely to be reported (published) if their findings were positive. The presence of small-study effects can be verified by visual inspection of the funnel plot, where for each included study of the meta-analysis, the estimate of the reported effect size is depicted against a measure of precision or sample size. The premise is that the scatter of plots should reflect a funnel shape, if small-study effects do not exist. However, when small studies are predominately in one direction (usually the direction of larger effect sizes), asymmetry will ensue.

The fat function implements several tests for detecting funnel plot asymmetry, which can be used when the presence of between-study heterogeneity in treatment effect is relatively low.

Usage

fat(b, b.se, n.total, d.total, d1, d2, method = "E-FIV")

Arguments

b	Vector with the effect size of each study. Examples are log odds ratio, log hazards ratio, log relative risk.
b.se	Optional vector with the standard error of the effect size of each study
n.total	Optional vector with the total sample size of each study
d.total	Optional vector with the total number of observed events for each study
d1	Optional vector with the total number of observed events in the exposed groups
d2	Optional vector with the total number of observed events in the unexposed groups
method	Method for testing funnel plot asymmetry, defaults to "E-FIV" (Egger's test with multiplicative dispersion). Other options are E-UW, M-FIV, M-FPV, D-FIV and D-FAV. More info in "Details"

Details

Egger regression method

A common method to test the presence of small-study effects is given as the following unweighted regression model (method="E-UW", Egger 1997):

b = B0 + B1*b.se + e; e~N(0, s^2)

Whereas B0 indicates the size and direction of the treatment effect, B1 provides a measure of asymmetry; the larger its deviation from zero the more pronounced the asymmetry. Otherwise, if B1=0, there is no association between the estimated effect sizes b and their corresponding estimates for the standard error b.se among the reported studies, indicating no asymmetry and thus no small-study effects.

It is possible to allow for between-study heterogeneity by adopting a multiplicative overdispersion parameter by which the variance in each study is multiplied (method="E-FIV", Sterne 2000):

b = B0 + B1*b.se + e; e~N(0, P*b.se^2)

Unfortunately, both tests are known to be intrinsically biased because: (i) the independent variable is subject to sampling variability; (ii) the standardized treatment effect is correlated with its estimated precision; and (iii) for binary data, the independent regression variable is a biased estimate of the true precision, with larger bias for smaller sample sizes (Macaskill 2001).

Macaskill regression method

To overcome the problems with the Egger approach, Macaskill et al. consider fitting a regression directly to the data using the treatment effect as the dependent variable, and study size (n.total) as the independent variable. Again, the observations are weighted by the inverse variance of the estimate to allow for possible heteroscedasticity (method="M-FIV", Macaskill 2001):

b = B0 + B1*n.total + e; e~N(0, P*b.se^2)

Macaskill et al. also proposed an alternative test where a 'pooled' estimate of the outcome proportion is used for the variance b.se^2 (method="M-FPV", Macaskill 2001):

b = B0 + B1*n.total + e; e~N(0, P/(d.total * (1-d.total/n.total)))

For studies with zero events, a continuity correction is applied by adding 0.5 to all cell counts.

Peters regression method

A modification of Macaskill's test was proposed by Peters et al. to obtain more balanced type-I error rates in the tail probability areas (method="P-FPV", Peters 2006):

b = B0 + B1/n.total + e; e~N(0, P/(d.total * (1-d.total/n.total)))

Again, 0.5 is added to all cells for studies with zero events.

Debray regression method

Because the use of aforementioned tests may be less appropriate in the presence of survival data, Debray et al. proposed using the total number of events (d.total) as independent variable (method="D-FIV", Debray 2017):

b = B0 + B1/d.total + e; e~N(0, P*b.se^2)

For studies with zero events, the total number of observed events is set to 1. Alternatively, when b.se is unknown or derived from small samples, Debray at al.proposed to use the following regression model (method="D-FAV", Debray 2017):

b = B0 + B1/d.total + e; e~N(0, P/(1/d1 + 1/d2))

Value

a list containing the following entries:

"pval"

A two-sided P-value indicating statistical significance of the funnel plot asymettry test. Values below the significance level (usually defined as 10%) support the presence of funnel plot asymmetry, and thus small-study effects.

"model"

A fitted glm object, representing the estimated regression model used for testing funnel plot asymmetry.

Author(s)

Thomas Debray <[e-mail]>

References

Debray TPA, Moons KGM, Riley RD. Detecting small-study effects and funnel plot asymmetry in meta-analysis of survival data: a comparison of new and existing tests. Res Syn Meth. 2018;9(1):41–50.

Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34.

Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta-analysis. Stat Med. 2001;20(4):641–54.

Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006 Feb 8;295(6):676–80.

Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol. 2000;53(11):1119–29.

See Also

plot.fat

Examples

data(Fibrinogen)
b <- log(Fibrinogen$HR)
b.se <- ((log(Fibrinogen$HR.975) - log(Fibrinogen$HR.025))/(2*qnorm(0.975)))
n.total <- Fibrinogen$N.total
d.total <- Fibrinogen$N.events

fat(b=b, b.se=b.se)
fat(b=b, b.se=b.se, d.total=d.total, method="D-FIV")

# Note that many tests are also available via metafor
require(metafor)
fat(b=b, b.se=b.se, n.total=n.total, method="M-FIV")
regtest(x=b, sei=b.se, ni=n.total, model="lm", predictor="ni") 

metamisc documentation built on Oct. 13, 2021, 3 p.m.