| yardstickSummary | R Documentation |
yardstickSummary() converts a yardstick::metric_set() into a summary
function that can be passed to the summaryFunction argument of
trainControl() (or the summary element of the functions argument of
rfeControl() and sbfControl()). This gives train() access to all of
the performance metrics in the yardstick package.
yardstickSummary(metrics)
metrics |
A metric set created by |
The names of the resulting performance estimates are the names of the
metrics (e.g. "mcc" or "brier_class"), and these should be used for the
metric argument of train().
For classification models, any metrics that require class probabilities
(such as roc_auc or brier_class) need classProbs = TRUE in
trainControl(). When there are two classes, the first factor level is
treated as the event of interest, consistent with twoClassSummary().
If case weights are passed to train(), they are also used to compute the
metrics.
By default, train() determines whether a metric should be maximized or
minimized using the yardstick metric's direction. A few metrics, such as
mpe and msd, are optimal when they are closest to zero. These can be
reported but should not be used to choose the tuning parameters.
A function with arguments data, lev, and model that returns a
named numeric vector of performance estimates.
postResample(), trainControl()
Other performance:
confusionMatrix(),
defaultSummary(),
negPredValue(),
recall()
library(yardstick)
reg_summary <- yardstickSummary(metric_set(rmse, ccc, huber_loss))
set.seed(3762)
dat <- data.frame(obs = rnorm(20))
dat$pred <- dat$obs + rnorm(20, sd = 0.5)
reg_summary(dat)
cls_summary <- yardstickSummary(metric_set(mcc, brier_class, roc_auc))
classes <- c("class1", "class2")
dat <- data.frame(
obs = factor(sample(classes, 50, replace = TRUE)),
pred = factor(sample(classes, 50, replace = TRUE)),
class1 = runif(50)
)
dat$class2 <- 1 - dat$class1
cls_summary(dat, lev = classes)
## Not run:
ctrl <- trainControl(
method = "cv",
classProbs = TRUE,
summaryFunction = cls_summary
)
train(Class ~ ., data = twoClassSim(200), method = "lda",
metric = "brier_class", trControl = ctrl)
## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.