vimpute: Impute missing values with prefered model, sequentially, with...

In VIM: Visualization and Imputation of Missing Values

Impute missing values with prefered model, sequentially, with hyperparametertuning and with PMM (if wanted)

Description

Impute missing values with prefered model, sequentially, with hyperparametertuning and with PMM (if wanted)

Usage

vimpute(
  data,
  ...,
  considered_variables = names(data),
  method = setNames(as.list(rep("ranger", length(considered_variables))),
    considered_variables),
  pmm = FALSE,
  pmm_k = NULL,
  pmm_k_method = "mean",
  learner_params = NULL,
  formula = FALSE,
  makeNA = NULL,
  donorcond = NULL,
  sequential = TRUE,
  nseq = 10,
  eps = 0.005,
  imp_var = TRUE,
  keep_all_columns = TRUE,
  pred_history = FALSE,
  tune = FALSE,
  verbose = FALSE,
  boot = NULL,
  robustboot = "stratified",
  uncert = "pmm",
  m = 1L,
  seed = NULL,
  tuned_params = NULL,
  tune_control = NULL,
  predictors = NULL,
  visit_sequence = "asis",
  spec = NULL
)

Arguments

data	Dataset with missing values. Provide as a data.table.
...	Optional bare grammar formulas, one per variable: target ~ predictors | method(...) – e.g. Sleep ~ Dream + Span | ranger(tune = TRUE) or NonD ~ . | robust(donorcond = ">= 0") – plus an optional .default = spec for unlisted variables. A plain-column right-hand side restricts the predictors (works for every method); a right-hand side with transformations (s(x), log(x), I(x^2), interactions) becomes a model formula (formula-capable methods only); . means all other variables. Grammar formulas compile to spec objects and cannot be combined with spec = or the flat per-variable arguments.
considered_variables	A character vector of variable names to be either imputed or used as predictors, excluding irrelevant columns from the imputation process. Excluded columns are still returned unchanged by default (see keep_all_columns).
method	Specifies the imputation method for each variable. Can be provided either: as a single global method (e.g. "ranger"), applied to all variables, or as a named list (e.g. list(var1 = "xgboost", var2 = "robust")), assigning a method to each variable individually. Built-in methods: ranger (Random Forest) xgboost (Gradient Boosting) regularized (glmnet regression/classification) robust (robustbase::lmrob / glmrob) gam (Generalized Additive Model via mgcv::gam) robgam (Robust GAM with outlier downweighting, simple or iterative reweighting) restricted (ECOSolveR least-squares regression with validate rules) Additional methods backed by any mlr3 learner pair can be added with register_vimpute_method(); vimpute_methods() lists everything currently registered.
pmm	Predictive Mean Matching (PMM) settings. Can be provided: as a single TRUE/FALSE (global), or as a named list, assigning PMM per (numeric) variable.
pmm_k	Number of nearest neighbors used in PMM. Accepted forms: single global integer (applies to all variables), or named list assigning values per variable, or NULL (default), meaning: k = 1 automatically for variables using PMM, k = NULL for variables without PMM
pmm_k_method	Aggregation method used when pmm_k > 1 in PMM. Default is "mean". Accepted forms: single global string ("mean", "median", "random"), or single global function (called with the k nearest observed values), or named list assigning methods per variable, or NULL values inside such lists, which fall back to "mean" Semantics: "mean": mean of the k nearest neighbors "median": median of the k nearest neighbors "random": random draw of one among the k nearest neighbors function: custom aggregator returning one numeric value
learner_params	Hyperparameters for the chosen methods. Can be provided in three ways: Per variable (e.g. list(mpg = list(num.trees = 500))) Per method (e.g. list(ranger = list(num.trees = 600))) Global, applied to all variables using the same method For restricted, set save_optimization_problem = TRUE to attach the exact ECOS problem arguments to the returned object under the restricted_optimization_problems attribute. Set robust = TRUE to replace least squares with Huber loss; huber_k controls its tuning constant and defaults to 1.345.
formula	Optional modeling formula to restrict or transform predictor variables. Only supported for methods whose registry entry declares formula support: among the built-ins regularized (glmnet), robust (lmrob/glmrob), gam (mgcv::gam), robgam (robust GAM), and restricted (ECOSolveR least-squares with validate rules) Provide as a named list, e.g.: list(mpg = mpg ~ hp + drat) list(hp = log(hp) ~ wt + cyl) For X: follows the rules of model.matrix For Y: transformations supported are log(), exp(), sqrt(), I(1/..). Only applicable for numeric variables.
makeNA	Optional named list that defines additional values to be treated as imputable missing values per variable, similar to kNN(). For variables listed in makeNA, only the specified values are imputed; existing NA values are left untouched. Variables not listed in makeNA continue to impute regular NA values.
donorcond	Optional named list of donor conditions per variable, similar to kNN(). Rows whose observed target values do not satisfy the condition are excluded from the donor pool for model fitting for that variable.
sequential	If TRUE, all variables with missing data are imputed sequentially across iterations.
nseq	Maximum number of iterations (if sequential is TRUE).
eps	Convergence threshold on the per-variable relative change between iterations: for numeric variables the mean squared change of the imputed values divided by the variance of the observed values, for factors the share of imputed cells whose category changed. The sequential process stops early once the largest per-variable change stays below eps for two consecutive iterations. The full iterations-by-variables change matrix is returned as attr(result, "convergence").
imp_var	If TRUE, additional columns indicating imputed values (VAR_imp) are added.
keep_all_columns	If TRUE (default), the full input is returned: columns excluded via considered_variables are passed through unchanged (original column order, with any VAR_imp indicators appended), matching kNN(), hotdeck() and irmi(). Set FALSE to return only the considered columns (plus their indicators), dropping the rest.
pred_history	If TRUE, all predicted values across all iterations are stored.
tune	Hyperparameter tuning flag. Can be: TRUE/FALSE globally or a list specifying tuning per variable, e.g. list(var1 = TRUE) Tuning runs once per variable, early in the iteration sequence. With m > 1, tuning runs once in the first imputation and the chosen parameters are shared by all m imputations (as in mice); the resulting vimmi object carries the tuning report in its tuning_log element.
verbose	If TRUE additional debugging output is provided
boot	If TRUE, bootstrap resampling is applied before model fitting to account for model uncertainty. Defaults to TRUE when m > 1 (each imputation then refits on a bootstrap sample, giving approximately proper multiple-imputation draws) and to FALSE for single imputation (m = 1); set explicitly to override. The bootstrap strategy is controlled by robustboot. Most effective with method = "robust". Default: FALSE
robustboot	Bootstrap strategy when boot = TRUE. Options: "standard" (classical bootstrap), "stratified" (good/bad residual split, default), "residual" (inverse residual weighting).
uncert	Imputation uncertainty method applied to numeric predictions: "pmm" (default since 7.3.0: predictive mean matching, Little 1988 – a random draw among the 5 donors whose predicted values are nearest the missing cell's prediction, so imputed values are observed values; for a no-bootstrap ranger fit the donors are scored by their out-of-bag predictions), "none" (deterministic point prediction; the pre-7.3.0 default), "normalerror" (add N(0, sigma_hat)), "resid" (add sampled residual), "midastouch" (covariate-distance-weighted PMM, Siddique & Belin 2008). Factor targets draw from the predicted class probabilities whenever uncert != "none" (and are imputed by the most probable class otherwise). If pmm = TRUE is set, it takes precedence over uncert. Variables imputed by method = "restricted" always use uncert = "none": a value-level draw on top of the constrained solution would break the validation rules the solver just enforced. The default is overridden silently, an explicitly requested draw mechanism warns.
m	Number of multiple imputations. Default: 1 (single imputation). When m > 1, returns a vimmi object storing the original data and imputed values efficiently. Use vim_complete to extract completed datasets.
seed	Optional single number for reproducibility. Applied once via set.seed at the start of the call (as in mice), so the whole run – including all m imputations – is reproducible while the m imputations still differ from each other. Default NULL leaves the random-number stream untouched.
tuned_params	Optional named list mapping variable names to learner parameter lists (e.g. list(Sleep = list(num.trees = 300L))). The parameters are applied to the variable's learner without running the tuner – use this to reuse tuning results across calls (each entry of a previous run's tuning_log carries its chosen parameters in $params). Used internally by m > 1 to share the first imputation's tuned parameters across all imputations.
tune_control	NULL (default) or a vimpute_tune_control object controlling the tuning of tune = TRUE: evaluation budget, CV folds of the tuning resampling, tuner and batch size. NULL keeps the built-in data-size heuristics. With m > 1 it applies to the single tuning run whose parameters all imputations share. The tuning_log records the budget and folds used per variable.
predictors	Optional per-variable predictor control, the equivalent of mice's predictorMatrix – and unlike formula it works for EVERY method, including ranger and xgboost. Either a named list mapping a target variable to the character vector of its predictors (e.g. list(Sleep = c("Dream", "Span"))), or a 0/1 (or logical) matrix with targets in rows and predictors in columns (compatible with mice::make.predictorMatrix). Variables without an entry use all other considered variables. A formula supplied for a variable takes precedence over its predictors entry (as in mice).
visit_sequence	Order in which the variables with missings are imputed: "asis" (default; column order), "increasing.na" (fewest missings first), "decreasing.na", or a character vector giving an explicit permutation of the NA-variables.
spec	NULL or a named list of vimpute_spec objects – one per variable, e.g. spec = list(Sleep = vs_ranger(num.trees = 300, tune = TRUE), NonD = vs_robust(donorcond = ">= 0"), .default = vs_ranger()). Each spec bundles the variable's method, learner parameters (validated eagerly), formula/predictors, tune, PMM settings, makeNA and donorcond; the reserved name ".default" covers unlisted variables. Compiles to the flat per-variable arguments, which therefore cannot be given in the same call.

Details

Missingness assumptions. Like all conditional (fully conditional specification) imputation, vimpute() assumes the data are MAR (missing at random: the probability of missingness may depend on observed values) – which includes MCAR (missing completely at random) as a special case. Under MNAR (missingness depending on the unobserved values themselves) imputations and downstream estimates can be biased, and no imputation method can fix this from the observed data alone; sensitivity analyses are advisable. makeMissing generates MCAR/MAR/MNAR missingness in complete data for exactly such simulation-based checks, and overimpute diagnoses the calibration of the imputation model on the observed cells.

Value

For m = 1: the imputed dataset, classed like the input (data.frame in, data.frame out; data.table in, data.table out). When tune = TRUE the tuning report is attached as attr(result, "tuning_log"); when pred_history = TRUE the prediction history is attached as attr(result, "pred_history"); sequential runs attach the per-variable convergence matrix as attr(result, "convergence") and the chain statistics as attr(result, "chain"); the per-variable model quality (NRMSE/PFC, out-of-bag for ranger, in-sample otherwise) is attached as attr(result, "model_error") – the return is always the data itself, never a wrapper list. For m > 1: a vimmi object.

Author(s)

Eileen Vattheuer, Matthias Templ, Alexander Kowarik

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, xgboostImpute()

Examples

## Not run: 
# Single imputation (default)
x <- vimpute(data = sleep, sequential = FALSE)

# Sequential imputation with 3 iterations
y <- vimpute(data = sleep, sequential = TRUE, nseq = 3)

# Impute only selected variables
z <- vimpute(data = sleep, considered_variables =
       c("Sleep", "Dream", "Span", "BodyWgt"), sequential = FALSE)

# Multiple imputation (m = 5) with bootstrap and residual uncertainty
# Returns a vimmi object
result <- vimpute(data = sleep, method = "ranger", sequential = FALSE,
                  imp_var = FALSE, m = 5, boot = TRUE, uncert = "resid")
print(result)

# Extract completed datasets
d1 <- vim_complete(result, 1)         # first imputed dataset
all_d <- vim_complete(result, "all")  # list of 5 datasets
long_d <- vim_complete(result, "long") # long format with .imp column

# Fit a model on each imputed dataset
fits <- with(result, lm(Sleep ~ Dream + Span))

# Multiple imputation with robust method and residual uncertainty
result2 <- vimpute(data = sleep, method = "robust", m = 5,
                   boot = TRUE, robustboot = "stratified",
                   uncert = "normalerror")

## End(Not run)

VIM documentation built on Sept. 2, 2026, 5:07 p.m.