pkonfound: Perform sensitivity analysis for published studies

In jrosen48/konfound: Quantify the Robustness of Causal Inferences

Perform sensitivity analysis for published studies

Description

For published studies, this command calculates (1) how much bias there must be in an estimate to nullify/sustain an inference; (2) the impact of an omitted variable necessary to nullify/sustain an inference for a regression coefficient. For a full description of the command’s usage and additional examples, please refer to our practical guide.

Usage

pkonfound(
  est_eff,
  std_err,
  n_obs,
  n_covariates = 1,
  alpha = 0.05,
  tails = 2,
  index = "RIR",
  nu = 0,
  n_treat = NULL,
  switch_trm = TRUE,
  model_type = "ols",
  a = NULL,
  b = NULL,
  c = NULL,
  d = NULL,
  two_by_two_table = NULL,
  test = "fisher",
  replace = "control",
  sdx = NA,
  sdy = NA,
  R2 = NA,
  far_bound = 0,
  eff_thr = NA,
  FR2max = 0,
  FR2max_multiplier = 1.3,
  to_return = "print",
  upper_bound = NULL,
  lower_bound = NULL,
  raw_treatment_success = NULL
)

Arguments

est_eff	the estimated effect (e.g., an unstandardized beta coefficient or a group mean difference).
std_err	the standard error of the estimate of the unstandardized regression coefficient.
n_obs	the number of observations in the sample.
n_covariates	the number of covariates in the regression model.
alpha	the probability of rejecting the null hypothesis (defaults to 0.05).
tails	integer indicating if the test is one-tailed (1) or two-tailed (2; defaults to 2).
index	specifies whether output is RIR or IT (impact threshold); defaults to "RIR".
nu	specifies the hypothesis to be tested; defaults to testing whether est_eff is significantly different from 0.
n_treat	the number of cases associated with the treatment condition (for logistic regression models).
switch_trm	indicates whether to switch the treatment and control cases; defaults to FALSE.
model_type	the type of model; defaults to "ols", but can be set to "logistic".
a	the number of cases in the control group showing unsuccessful results (2x2 table model).
b	the number of cases in the control group showing successful results (2x2 table model).
c	the number of cases in the treatment group showing unsuccessful results (2x2 table model).
d	the number of cases in the treatment group showing successful results (2x2 table model).
two_by_two_table	a table (matrix, data.frame, tibble, etc.) from which a, b, c, and d can be extracted.
test	specifies whether to use Fisher's Exact Test ("fisher") or a chi-square test ("chisq"); defaults to "fisher".
replace	specifies whether to use the entire sample ("entire") or the control group ("control") for calculating the base rate; default is "control".
sdx	the standard deviation of X (used for unconditional ITCV).
sdy	the standard deviation of Y (used for unconditional ITCV).
R2	the unadjusted, original R^2 in the observed function (used for unconditional ITCV).
far_bound	indicates whether the estimated effect is moved to the boundary closer (0, default) or further away (1).
eff_thr	for RIR: the unstandardized coefficient threshold to change an inference; for IT: the correlation defining the threshold for inference.
FR2max	the largest R^2 (or R^2_{\max}) in the final model with an unobserved confounder (used for COP).
FR2max_multiplier	the multiplier applied to R^2 to derive R^2_{\max}; defaults to 1.3 (used for COP).
to_return	specifies the output format: "print" (default) to display output, "plot" for a plot, or "raw_output" to return a data.frame for further analysis.
upper_bound	optional (replaces est_eff); the upper bound of the confidence interval.
lower_bound	optional (replaces est_eff); the lower bound of the confidence interval.
raw_treatment_success	optional; the unadjusted count of successful outcomes in the treatment group for calculating the specific RIR benchmark.

Details

The function accepts arguments depending on the type of model:

Linear Models (index: RIR, ITCV, PSE, COP)

• est_eff, std_err, n_obs, n_covariates, alpha, tails, index, nu

• sdx, sdy, R2, far_bound, eff_thr, FR2max, FR2max_multiplier

• upper_bound, lower_bound

Logistic Regression Model

• est_eff, std_err, n_obs, n_covariates, n_treat, alpha, tails, nu

• replace, switch_trm, raw_treatment_success, model_type

2x2 Table Model (Non-linear)

• a, b, c, d, two_by_two_table, test, replace, switch_trm

Values

pkonfound prints the bias and the number of cases that would have to be replaced with cases for which there is no effect to nullify the inference. If to_return = "raw_output", a list is returned with the following components:

RIR & ITCV for linear model

obs_r

correlation between predictor of interest (X) and outcome (Y) in the sample data

act_r

correlation between predictor of interest (X) and outcome (Y) from the sample regression based on the t-ratio accounting for non-zero null hypothesis

critical_r

critical correlation value at which the inference would be nullified (e.g., associated with p=.05)

r_final

final correlation value given confounding variable (CV). Should be equal to critical_r

rxcv

unconditional corr(X,CV) necessary to nullify the inference for smallest impact

rycv

unconditional corr(Y,CV) necessary to nullify the inference for smallest impact

rxcvGz

corr(X,CV|Z) conditioning on all observed covariates

rycvGz

corr(Y,CV|Z) conditioning on all observed covariates

itcv

unconditional ITCV (uncond_rxcv * uncond_rycv)

itcvGz

conditional ITCV given all observed covariates

r2xz

R^2 using all observed covariates to explain the predictor of interest (X)

r2yz

R^2 using all observed covariates to explain the predictor of interest (Y)

beta_threshold

threshold for for estimated effect

beta_threshold_verify

verified threshold matching beta_threshold

perc_bias_to_change

percent bias to change inference

RIR_primary

Robustness of Inference to Replacement (RIR)

RIR_supplemental

RIR for an extra row or column that is needed to nullify the inference

RIR_perc

RIR as % of total sample (for linear regression) or as % of data points in the cell where replacement takes place (for logistic and 2 by 2 table)

Fig_ITCV

ITCV plot object

Fig_RIR

RIR threshold plot object

COP for linear model

delta*

delta calculated using Oster’s unrestricted estimator

delta*restricted

delta calculated using Oster’s restricted estimator

delta_exact

delta calculated using correlation-based approach

delta_pctbias

percent bias when comparing delta* to delta_exact

var(Y)

variance of the dependent variable (\sigma_Y^2)

var(X)

variance of the independent variable (\sigma_X^2)

var(CV)

variance of the confounding variable (\sigma_{CV}^2)

cor_oster

correlation matrix implied by delta*

cor_exact

correlation matrix implied by delta_exact

eff_x_M3_oster

effect estimate for X under the Oster‑PSE variant

eff_x_M3

effect estimate for X under the PSE adjustment

Table

formatted results table

Figure

COP diagnostic plot

PSE for linear model

corr(X,CV|Z)

correlation between X and CV conditional on Z

corr(Y,CV|Z)

correlation between Y and CV conditional on Z

corr(X,CV)

correlation between X and CV

corr(Y,CV)

correlation between X and CV

covariance matrix

covariance matrix among Y, X, Z, and CV under the PSE adjustment

eff_M3

estimated unstandardized regression coefficient for X in M3 under the PSE adjustment

se_M3

standard error of that coefficient in M3 under the PSE adjustment

Table

matrix summarizing key statistics from three nested regression models (M1, M2, M3)

RIR for logistic model

RIR_primary

Robustness of Inference to Replacement (RIR)

RIR_supplemental

RIR for an extra row or column that is needed to nullify the inference

RIR_perc

RIR as % of data points in the cell where replacement takes place

fragility_primary

Fragility; the number of switches (e.g., treatment success to treatment failure) to nullify the inference

fragility_supplemental

Fragility for an extra row or column that is needed to nullify the inference

starting_table

observed (implied) 2 by 2 table before replacement and switching

final_table

the 2 by 2 table after replacement and switching

user_SE

user-entered standard error

analysis_SE

the standard error used to generate a plausible 2 by 2 table

needtworows

indicator whether extra switches were needed

RIR for 2×2 table model

RIR_primary

Robustness of Inference to Replacement (RIR)

RIR_supplemental

RIR for an extra row or column that is needed to nullify the inference

RIR_perc

RIR as % of data points in the cell where replacement takes place

fragility_primary

Fragility; the number of switches (e.g., treatment success to treatment failure) to nullify the inference

fragility_supplemental

Fragility for an extra row or column that is needed to nullify the inference

starting_table

observed 2 by 2 table before replacement and switching

final_table

the 2 by 2 table after replacement and switching

needtworows

indicator whether extra switches were needed

Note

For a thoughtful background on benchmark options for ITCV, see Cinelli & Hazlett (2020), Lonati & Wulff (2024), and Frank (2000).

Examples

## Linear models
pkonfound(2, .4, 100, 3)
pkonfound(-2.2, .65, 200, 3)
pkonfound(.5, 3, 200, 3)
# using a confidence interval
pkonfound(upper_bound = 3, lower_bound = 1, n_obs = 100, n_covariates = 3)   

## Plot examples
pkonfound(2, .4, 100, 3, to_return = "thresh_plot")
pkonfound(2, .4, 100, 3, to_return = "corr_plot")

## Logistic regression model example
pkonfound(-0.2, 0.103, 20888, 3, n_treat = 17888, model_type = "logistic")

## 2x2 table examples
pkonfound(a = 35, b = 17, c = 17, d = 38)
pkonfound(a = 35, b = 17, c = 17, d = 38, alpha = 0.01)
pkonfound(a = 35, b = 17, c = 17, d = 38, alpha = 0.01, switch_trm = FALSE)
pkonfound(a = 35, b = 17, c = 17, d = 38, test = "chisq")

## Advanced examples
# Calculating unconditional ITCV and benchmark correlation for ITCV
pkonfound(est_eff = .5, std_err = .056, n_obs = 6174, sdx = 0.22, sdy = 1, R2 = .3,
          index = "IT", to_return = "print")
# Calculating delta* and delta_exact 
pkonfound(est_eff = .4, std_err = .1, n_obs = 290, sdx = 2, sdy = 6, R2 = .7,
         eff_thr = 0, FR2max = .8, index = "COP", to_return = "raw_output")
# Calculating rxcv and rycv when preserving standard error
pkonfound(est_eff = .5, std_err = .056, n_obs = 6174, eff_thr = .1,
         sdx = 0.22, sdy = 1, R2 = .3, index = "PSE", to_return = "raw_output")

jrosen48/konfound documentation built on June 11, 2025, 2:46 p.m.