sim_dgp_scenarios: Simulation DGPs for Feature Importance Method Comparison

In xplainfi: Feature Importance Methods for Global Explanations

Simulation DGPs for Feature Importance Method Comparison

Description

These data generating processes (DGPs) are designed to illustrate specific strengths and weaknesses of different feature importance methods like PFI, CFI, and RFI. Each DGP focuses on one primary challenge to make the differences between methods clear.

Usage

sim_dgp_correlated(n = 500L, r = 0.9)

sim_dgp_mediated(n = 500L)

sim_dgp_confounded(n = 500L, hidden = TRUE)

sim_dgp_interactions(n = 500L)

sim_dgp_independent(n = 500L)

Arguments

n	(integer(1): 500L) Number of observations to generate.
r	(numeric(1): 0.9) Correlation between x1 and x2. Must be between -1 and 1.
hidden	(logical(1): TRUE) Whether to hide the confounder from the returned task. If FALSE, the confounder is included as a feature, allowing direct adjustment. If TRUE (default), only the proxy is available, simulating unmeasured confounding.

Details

Correlated Features DGP: This DGP creates highly correlated predictors where PFI will show artificially low importance due to redundancy, while CFI will correctly identify each feature's conditional contribution.

Mathematical Model:

(X_1, X_2)^T \sim \text{MVN}(0, \Sigma)

where \Sigma is a 2 \times 2 covariance matrix with 1 on the diagonal and correlation r on the off-diagonal.

X_3 \sim N(0,1), \quad X_4 \sim N(0,1)

Y = 2 \cdot X_1 + X_3 + \varepsilon

where \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• x1: Standard normal from MVN, direct causal effect on y (\beta = 2.0)

• x2: Correlated with x1 (correlation = r), NO causal effect on y (\beta = 0)

• x3: Independent standard normal, direct causal effect on y (\beta = 1.0)

• x4: Independent standard normal, no effect on y (\beta = 0)

Expected Behavior:

• Will depend on the used learner and the strength of correlation (r)

• Marginal methods (PFI, Marginal SAGE): Should falsely assign importance to x2 due to correlation with x1

• CFI Should correctly assign near-zero importance to x2

• x2 is a "spurious predictor" - correlated with causal feature but not causal itself

Mediated Effects DGP: This DGP demonstrates the difference between total and direct causal effects. Some features affect the outcome only through mediators.

Mathematical Model:

\text{exposure} \sim N(0,1), \quad \text{direct} \sim N(0,1)

\text{mediator} = 0.8 \cdot \text{exposure} + 0.6 \cdot \text{direct} + \varepsilon_m

Y = 1.5 \cdot \text{mediator} + 0.5 \cdot \text{direct} + \varepsilon

where \varepsilon_m \sim N(0, 0.3^2) and \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• exposure: Has no direct effect on y, only through mediator (total effect = 1.2)

• mediator: Mediates the effect of exposure on y

• direct: Has both direct effect on y and effect on mediator

• noise: No causal relationship to y

Causal Structure: exposure -> mediator -> y <- direct -> mediator

Confounding DGP: This DGP includes a confounder that affects both a feature and the outcome. Uses simple coefficients for easy interpretation.

Mathematical Model:

H \sim N(0,1)

X_1 = H + \varepsilon_1

\text{proxy} = H + \varepsilon_p, \quad \text{independent} \sim N(0,1)

Y = H + X_1 + \text{independent} + \varepsilon

where all \varepsilon \sim N(0, 0.5^2) independently.

Model Structure:

• Confounder H ~ N(0,1) (potentially unobserved)

• x1 = H + noise (affected by confounder)

• proxy = H + noise (noisy measurement of confounder)

• independent ~ N(0,1) (truly independent)

• y = H + x1 + independent + noise

Expected Behavior:

• PFI: Will show inflated importance for x1 due to confounding

• CFI: Should partially account for confounding through conditional sampling and reduce its importance

• RFI conditioning on proxy: Should reduce confounding bias by conditioning on proxy

Interaction Effects DGP: This DGP demonstrates a pure interaction effect where features have no main effects.

Mathematical Model:

Y = 2 \cdot X_1 \cdot X_2 + X_3 + \varepsilon

where X_j \sim N(0,1) independently and \varepsilon \sim N(0, 0.5^2).

Feature Properties:

• x1, x2: Independent features with ONLY interaction effect (no main effects)

• x3: Independent feature with main effect only

• noise1, noise2: No causal effects

Expected Behavior:

• Will depend on the used learner and its ability to model interactions

Independent Features DGP: This is a baseline scenario where all features are independent and their effects are additive. All importance methods should give similar results.

Mathematical Model:

Y = 2.0 \cdot X_1 + 1.0 \cdot X_2 + 0.5 \cdot X_3 + \varepsilon

where X_j \sim N(0,1) independently and \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• important1-3: Independent features with different effect sizes

• unimportant1-2: Independent noise features with no effect

Expected Behavior:

• All methods: Should rank features consistently by their true effect sizes

• Ground truth: important1 > important2 > important3 > unimportant1,2 (approximately 0)

Value

A regression task (mlr3::TaskRegr) with data.table backend.

Functions

• sim_dgp_correlated(): Correlated features demonstrating PFI's limitations

• sim_dgp_mediated(): Mediated effects showing direct vs total importance

• sim_dgp_confounded(): Confounding scenario for conditional sampling

• sim_dgp_interactions(): Interaction effects between features

• sim_dgp_independent(): Independent features baseline scenario

References

Ewald F, Bothmann L, Wright M, Bischl B, Casalicchio G, König G (2024). “A Guide to Feature Importance Methods for Scientific Inference.” In Longo L, Lapuschkin S, Seifert C (eds.), Explainable Artificial Intelligence, 440–464. ISBN 978-3-031-63797-1, \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/978-3-031-63797-1_22")}.

See Also

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Examples

task = sim_dgp_correlated(200)
task$data()

# With different correlation
task_high_cor = sim_dgp_correlated(200, r = 0.95)
cor(task_high_cor$data()$x1, task_high_cor$data()$x2)
task = sim_dgp_mediated(200)
task$data()
# Hidden confounder scenario (traditional)
task_hidden = sim_dgp_confounded(200, hidden = TRUE)
task_hidden$feature_names  # proxy available but not confounder

# Observable confounder scenario
task_observed = sim_dgp_confounded(200, hidden = FALSE)
task_observed$feature_names  # both confounder and proxy available
task = sim_dgp_interactions(200)
task$data()
task = sim_dgp_independent(200)
task$data()

xplainfi documentation built on Feb. 27, 2026, 1:08 a.m.