vimp_hypothesis_test: Perform a hypothesis test against the null hypothesis of...

In bdwilliamson/nova: Perform Inference on Algorithm-Agnostic Variable Importance

Perform a hypothesis test against the null hypothesis of \delta importance

Description

Perform a hypothesis test against the null hypothesis of zero importance by: (i) for a user-specified level \alpha, compute a (1 - \alpha)\times 100% confidence interval around the predictiveness for both the full and reduced regression functions (these must be estimated on independent splits of the data); (ii) if the intervals do not overlap, reject the null hypothesis.

Usage

vimp_hypothesis_test(
  predictiveness_full,
  predictiveness_reduced,
  se,
  delta = 0,
  alpha = 0.05
)

Arguments

predictiveness_full	the estimated predictiveness of the regression including the covariate(s) of interest.
predictiveness_reduced	the estimated predictiveness of the regression excluding the covariate(s) of interest.
se	the estimated standard error of the variable importance estimator
delta	the value of the \delta-null (i.e., testing if importance < \delta); defaults to 0.
alpha	the desired type I error rate (defaults to 0.05).

Details

See the paper by Williamson, Gilbert, Simon, and Carone for more details on the mathematics behind this function and the definition of the parameter of interest.

Value

a list, with: the hypothesis testing decision (TRUE if the null hypothesis is rejected, FALSE otherwise); the p-value from the hypothesis test; and the test statistic from the hypothesis test.

bdwilliamson/nova documentation built on Feb. 1, 2024, 10:04 p.m.