dgm.Carter2019: Carter et al. (2019) Data-Generating Mechanism

In PublicationBiasBenchmark: Benchmark for Publication Bias Correction Methods

Carter et al. (2019) Data-Generating Mechanism

Description

This data-generating mechanism simulates primary studies estimating treatment effects using Cohen's d. The observed effect size is modeled as a fixed mean plus random heterogeneity across studies, with sample sizes varying to generate differences in standard errors. The simulation introduces publication bias via a selection algorithm where the probability of publication depends nonlinearly on the sign and p-value of the effect, with regimes for no, medium, and strong publication bias. It also incorporates questionable research practices (QRPs) such as optional outlier removal, selection between dependent variables, use of moderators, and optional stopping.

The description and code is based on \insertCitehong2021using;textualPublicationBiasBenchmark. The data-generating mechanism was introduced in \insertCitecarter2019correcting;textualPublicationBiasBenchmark.

Usage

## S3 method for class 'Carter2019'
dgm(dgm_name, settings)

Arguments

dgm_name	DGM name (automatically passed)
settings	List containing mean_effect Mean effect effect_heterogeneity Mean effect heterogeneity bias Degree of publication bias with one of following levels: "none", "medium", "high". QRP Degree of questionable research practices with one of following levels: "none", "medium", "high". n_studies Number of effect size estimates

Details

This simulation environment is based on the framework described by Carter, Schönbrodt, Gervais, and Hilgard (2019). In this setup, primary studies estimate the effect of a treatment using Cohen's d as the effect size metric. The observed difference between treatment and control groups is modeled as the sum of a fixed effect (alpha1) and a random component, which introduces effect heterogeneity across studies. The degree of heterogeneity is controlled by the parameter sigma2_h. Variability in the standard errors of d is generated by simulating primary studies with different sample sizes.

The simulation incorporates two main types of distortions in the research environment. First, a publication selection algorithm is used, where the probability of a study being "published" depends nonlinearly on both the sign of the estimated effect and its p-value. Three publication selection regimes are modeled: "No Publication Bias," "Medium Publication Bias," and "Strong Publication Bias," each defined by different parameters in the selection algorithm. Second, the simulation includes four types of questionable research practices (QRPs): (a) optional removal of outliers, (b) optional selection between two dependent variables, (c) optional use of moderators, and (d) optional stopping.

Value

Data frame with

yi

effect size

sei

standard error

ni

sample size

es_type

effect size type

Author(s)

František Bartoš [e-mail] (adapted from Hong and Reed 2021)

References

\insertAllCited

See Also

dgm(), validate_dgm_setting()

PublicationBiasBenchmark documentation built on March 16, 2026, 5:07 p.m.