classification_stability: Stability Index for Classification Models

In TrustworthyMLR: Stability and Robustness Evaluation for Machine Learning Models

Stability Index for Classification Models

Description

Computes the stability of classification predictions across multiple runs. For classification, stability is measured as the average agreement between pairs of runs, adjusted for chance (similar to Cohen's Kappa but extended for multiple runs).

Usage

classification_stability(class_matrix)

Arguments

class_matrix

A matrix or data.frame where each row represents an observation and each column represents a predicted class (factor or character) from a single model run.

Value

A numeric scalar between 0 and 1, where 1 indicates perfect consistency and 0 indicates consistency no better than chance.

Examples

# Simulate classification predictions from 3 runs
preds <- data.frame(
    run1 = c("A", "A", "B", "C"),
    run2 = c("A", "B", "B", "C"),
    run3 = c("A", "A", "B", "C")
)
classification_stability(preds)

TrustworthyMLR documentation built on Feb. 20, 2026, 5:09 p.m.