modelPrune: Model-Based Predictor Pruning

In corrselect: Correlation-Based and Model-Based Predictor Pruning

Model-Based Predictor Pruning

Description

modelPrune() performs iterative removal of fixed-effect predictors based on model diagnostics (e.g., VIF) until all remaining predictors satisfy a specified threshold. It supports linear models, generalized linear models, and mixed models.

Usage

modelPrune(
  formula,
  data,
  engine = "lm",
  criterion = "vif",
  limit = 5,
  force_in = NULL,
  max_steps = NULL,
  ...
)

Arguments

formula	A model formula specifying the response and predictors. May include random effects for mixed models (e.g., y ~ x1 + x2 + (1|group)).
data	A data.frame containing the variables in the formula.
engine	Either a character string for built-in engines, or a list defining a custom engine. Built-in engines (character string): "lm" (default): Linear models via stats::lm() "glm": Generalized linear models via stats::glm() (requires family argument) "lme4": Mixed models via lme4::lmer() or lme4::glmer() (requires lme4 package) "glmmTMB": Generalized linear mixed models via glmmTMB::glmmTMB() (requires glmmTMB package) Custom engine (named list with required components): fit: function(formula, data, ...) that returns a fitted model object diagnostics: function(model, fixed_effects) that returns a named numeric vector of diagnostic scores (one per fixed effect, higher values = worse) name (optional): character string used in error messages (default: "custom")
criterion	Character string specifying the diagnostic criterion for pruning. For built-in engines, supported values are: "vif" (default): Variance Inflation Factor. Measures how much the variance of a coefficient is inflated due to collinearity. Values > 5-10 indicate problematic multicollinearity. "condition_number": Condition index of the principal directions that carry a predictor's coefficient variance, from a singular value decomposition of the design matrix (see Details). Runs from 1 up to the overall condition number; higher values indicate greater collinearity. For custom engines, this parameter is ignored (diagnostics are computed by the engine's diagnostics function).
limit	Numeric scalar. Maximum allowed value for the criterion. Predictors with diagnostic values exceeding this limit are iteratively removed. Default: 5 (common VIF threshold).
force_in	Character vector of predictor names that must be retained in the final model. These variables will not be removed during pruning. Default: NULL.
max_steps	Integer. Maximum number of pruning iterations. If NULL (default), pruning continues until all diagnostics are below the limit or no more removable predictors remain.
...	Additional arguments passed to the modeling function (e.g., family for glm/glmer, control parameters for lme4/glmmTMB).

Details

modelPrune() works by:

1. Parsing the formula to identify fixed-effect predictors

2. Fitting the initial model

3. Computing diagnostics for each fixed-effect predictor

4. Checking feasibility of force_in constraints

5. Iteratively removing the predictor with the worst diagnostic value (excluding force_in variables) until all diagnostics <= limit

6. Returning the pruned data frame

Random Effects: For mixed models (lme4, glmmTMB), only fixed-effect predictors are considered for pruning. Random-effect structure is preserved exactly as specified in the original formula.

VIF Computation: Variance Inflation Factors are computed from the fixed-effects design matrix. For categorical predictors, VIF represents the inflation for the entire factor (not individual dummy variables).

Condition Number Computation: The fixed-effects design matrix is centered and scaled (scale(X, center = TRUE, scale = TRUE)) and decomposed by SVD. This is a common convention for collinearity screening, but it means the diagnostic does not reflect intercept-related collinearity the way an uncentered decomposition would.

A condition index belongs to a principal direction of the design matrix, not to a column, so each predictor is scored through the variance decomposition of Belsley, Kuh and Welsch. Direction k has condition index \eta_k = d_{\max}/d_k and carries a share \pi_{jk} = (v_{jk}^2/d_k^2) / \sum_m (v_{jm}^2/d_m^2) of predictor j's coefficient variance. The predictor's score is \sum_k \pi_{jk}\,\eta_k, the condition index of the directions its coefficient variance actually sits on. It runs from 1, for a predictor whose variance sits on the best-conditioned direction, up to the overall condition number for one whose variance sits on the worst, so limit is read on the familiar condition-index scale. A predictor loading on a numerically singular direction scores Inf.

The score reflects where a predictor's coefficient variance sits rather than how large it is, so it stays on the condition-index scale rather than the variance-ratio scale of the VIF. limit is therefore read differently under the two criteria and the same numeric value does not mean the same thing for both.

For a categorical predictor with multiple dummy columns, the score is the maximum across its columns rather than a joint decomposition restricted to that factor's subspace.

Determinism: The algorithm is deterministic. Ties in diagnostic values are broken by removing the predictor that appears last in the formula.

Force-in Constraints: If variables in force_in violate the diagnostic threshold, the function will error. This ensures that the constraint is feasible before pruning begins.

Value

A data.frame containing only the retained fixed-effect predictors (and response), plus any random-effect grouping/slope variables for mixed-model engines. The result has the following attributes:

selected_vars

Character vector of retained predictor names

removed_vars

Character vector of removed predictor names (in order of removal)

engine

Character string indicating which engine was used (for custom engines, this is the engine's name field)

criterion

Character string indicating which criterion was used

limit

The threshold value used

final_model

The final fitted model object (optional)

References

Belsley, D. A., Kuh, E. and Welsch, R. E. (1980) Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York.

See Also

corrPrune for association-based predictor pruning, corrSelect for exhaustive subset enumeration.

Examples

# Linear model with VIF-based pruning
data(mtcars)
pruned <- modelPrune(mpg ~ ., data = mtcars, engine = "lm", limit = 5)
names(pruned)

# Force certain predictors to remain
pruned <- modelPrune(mpg ~ ., data = mtcars, force_in = "drat", limit = 20)

# GLM example (requires family argument). Uses a small predictor set
# that avoids quasi-complete separation on this data (unlike `am ~ .`,
# which regresses a 32-row binary response on all 10 mtcars predictors).
pruned <- modelPrune(vs ~ mpg + wt + hp, data = mtcars, engine = "glm",
                     family = binomial(), limit = 5)

## Not run: 
# Custom engine example (INLA)
inla_engine <- list(
  name = "inla",
  fit = function(formula, data, ...) {
    inla::inla(formula = formula, data = data,
               family = list(...)$family %||% "gaussian",
               control.compute = list(config = TRUE))
  },
  diagnostics = function(model, fixed_effects) {
    scores <- model$summary.fixed[, "sd"]
    names(scores) <- rownames(model$summary.fixed)
    scores[fixed_effects]
  }
)

pruned <- modelPrune(y ~ x1 + x2 + x3, data = df,
                     engine = inla_engine, limit = 0.5)

## End(Not run)

corrselect documentation built on Aug. 23, 2026, 1:06 a.m.