metrics: Regression and classification metrics.

In funcml: Functional Machine Learning Framework

Regression and classification metrics.

Description

Base R implementations used across evaluation and interpretation utilities.

Usage

rmse(truth, pred)

mae(truth, pred)

mse(truth, pred)

rsq(truth, pred)

medae(truth, pred)

mape(truth, pred)

logloss(truth, prob_matrix)

brier(truth, prob_matrix)

accuracy(truth, pred_class)

precision(truth, pred_class)

recall(truth, pred_class)

specificity(truth, pred_class)

f1(truth, pred_class)

balanced_accuracy(truth, pred_class)

auc(truth, prob, average = c("macro", "weighted"))

auc_weighted(truth, prob)

calibration_curve(
  truth,
  prob,
  bins = 10,
  strategy = c("quantile", "uniform"),
  positive = NULL
)

ece(
  truth,
  prob,
  bins = 10,
  strategy = c("quantile", "uniform"),
  positive = NULL
)

mce(
  truth,
  prob,
  bins = 10,
  strategy = c("quantile", "uniform"),
  positive = NULL
)

Arguments

truth	Observed outcomes.
pred	Predicted numeric values or class labels.
prob_matrix	Matrix or vector of predicted probabilities (classification).
pred_class	Predicted class labels (classification).
prob	Probability vector (binary) or probability matrix with one column per class (multiclass).
average	For multiclass AUC, aggregation mode: "macro" or "weighted" (class-frequency weighted one-vs-rest AUC). Ignored for binary AUC.
bins	Number of bins for calibration summaries.
strategy	Binning strategy: "quantile" or "uniform".
positive	Optional positive/event class for binary classification.

Value

Numeric scalar metric.

Examples

truth_reg <- c(3, 5, 2.5, 7)
pred_reg <- c(2.8, 4.9, 2.7, 6.8)
rmse(truth_reg, pred_reg)
mae(truth_reg, pred_reg)
mse(truth_reg, pred_reg)
rsq(truth_reg, pred_reg)
medae(truth_reg, pred_reg)
mape(truth_reg, pred_reg)

truth_cls <- factor(c("no", "yes", "yes", "no"), levels = c("no", "yes"))
pred_cls <- factor(c("no", "yes", "no", "no"), levels = levels(truth_cls))
prob_cls <- cbind(
  no = c(0.8, 0.2, 0.6, 0.7),
  yes = c(0.2, 0.8, 0.4, 0.3)
)
logloss(truth_cls, prob_cls)
brier(truth_cls, prob_cls)
accuracy(truth_cls, pred_cls)
precision(truth_cls, pred_cls)
recall(truth_cls, pred_cls)
specificity(truth_cls, pred_cls)
f1(truth_cls, pred_cls)
balanced_accuracy(truth_cls, pred_cls)
auc(truth_cls, prob_cls[, "yes"])

truth_multi <- factor(c("a", "b", "c", "a", "b", "c"), levels = c("a", "b", "c"))
prob_multi <- rbind(
  c(0.90, 0.05, 0.05),
  c(0.05, 0.90, 0.05),
  c(0.05, 0.05, 0.90),
  c(0.85, 0.10, 0.05),
  c(0.10, 0.80, 0.10),
  c(0.05, 0.10, 0.85)
)
colnames(prob_multi) <- levels(truth_multi)
auc(truth_multi, prob_multi)
auc_weighted(truth_multi, prob_multi)
calibration_curve(truth_cls, prob_cls[, "yes"])
ece(truth_cls, prob_cls[, "yes"])
mce(truth_cls, prob_cls[, "yes"])

funcml documentation built on Aug. 22, 2026, 5:08 p.m.