stability_index: Stability Index for Model Predictions

In TrustworthyMLR: Stability and Robustness Evaluation for Machine Learning Models

Stability Index for Model Predictions

Description

Computes a Stability Index that quantifies the consistency of machine learning model predictions across multiple runs or resamples. A stability index of 1 indicates perfectly consistent predictions, while values closer to 0 indicate high variability across runs.

Usage

stability_index(predictions_matrix)

Arguments

predictions_matrix

A numeric matrix or data.frame where each row represents an observation and each column represents predictions from a single model run or resample. Must contain at least two columns and no missing values.

Details

The index is calculated by comparing the mean per-observation variance across runs to the overall variance of all predictions. Low per-observation variance relative to overall variance indicates that the model produces consistent results regardless of the specific training run or resample.

Value

A numeric scalar between 0 and 1, where 1 indicates perfect stability (identical predictions across all runs) and values near 0 indicate high instability.

Examples

# Simulate predictions from 5 model runs for 100 observations
set.seed(42)
base_predictions <- rnorm(100)
predictions <- matrix(
  rep(base_predictions, 5) + rnorm(500, sd = 0.1),
  ncol = 5
)
stability_index(predictions)

# Perfectly stable predictions yield an index of 1
stable_preds <- matrix(rep(1:10, 3), ncol = 3)
stability_index(stable_preds)

TrustworthyMLR documentation built on Feb. 20, 2026, 5:09 p.m.