yardstickSummary: Use yardstick metrics to measure performance

View source: R/yardstick.R

yardstickSummaryR Documentation

Use yardstick metrics to measure performance

Description

yardstickSummary() converts a yardstick::metric_set() into a summary function that can be passed to the summaryFunction argument of trainControl() (or the summary element of the functions argument of rfeControl() and sbfControl()). This gives train() access to all of the performance metrics in the yardstick package.

Usage

yardstickSummary(metrics)

Arguments

metrics

A metric set created by yardstick::metric_set().

Details

The names of the resulting performance estimates are the names of the metrics (e.g. "mcc" or "brier_class"), and these should be used for the metric argument of train().

For classification models, any metrics that require class probabilities (such as roc_auc or brier_class) need classProbs = TRUE in trainControl(). When there are two classes, the first factor level is treated as the event of interest, consistent with twoClassSummary().

If case weights are passed to train(), they are also used to compute the metrics.

By default, train() determines whether a metric should be maximized or minimized using the yardstick metric's direction. A few metrics, such as mpe and msd, are optimal when they are closest to zero. These can be reported but should not be used to choose the tuning parameters.

Value

A function with arguments data, lev, and model that returns a named numeric vector of performance estimates.

See Also

postResample(), trainControl()

Other performance: confusionMatrix(), defaultSummary(), negPredValue(), recall()

Examples

library(yardstick)

reg_summary <- yardstickSummary(metric_set(rmse, ccc, huber_loss))

set.seed(3762)
dat <- data.frame(obs = rnorm(20))
dat$pred <- dat$obs + rnorm(20, sd = 0.5)
reg_summary(dat)

cls_summary <- yardstickSummary(metric_set(mcc, brier_class, roc_auc))

classes <- c("class1", "class2")
dat <- data.frame(
  obs = factor(sample(classes, 50, replace = TRUE)),
  pred = factor(sample(classes, 50, replace = TRUE)),
  class1 = runif(50)
)
dat$class2 <- 1 - dat$class1
cls_summary(dat, lev = classes)

## Not run: 
ctrl <- trainControl(
  method = "cv",
  classProbs = TRUE,
  summaryFunction = cls_summary
)
train(Class ~ ., data = twoClassSim(200), method = "lda",
      metric = "brier_class", trControl = ctrl)

## End(Not run)

caret documentation built on Nov. 15, 2026, 9:06 a.m.