cal_validate_linear: Measure performance with and without using linear regression...

In topepo/probably: Tools for Post-Processing Predicted Values

Measure performance with and without using linear regression calibration

Description

Measure performance with and without using linear regression calibration

Usage

cal_validate_linear(
  .data,
  truth = NULL,
  estimate = dplyr::starts_with(".pred"),
  metrics = NULL,
  save_pred = FALSE,
  ...
)

## S3 method for class 'resample_results'
cal_validate_linear(
  .data,
  truth = NULL,
  estimate = dplyr::starts_with(".pred"),
  metrics = NULL,
  save_pred = FALSE,
  ...
)

## S3 method for class 'rset'
cal_validate_linear(
  .data,
  truth = NULL,
  estimate = dplyr::starts_with(".pred"),
  metrics = NULL,
  save_pred = FALSE,
  ...
)

Arguments

.data	An rset object or the results of tune::fit_resamples() with a .predictions column.
truth	The column identifier for the true class results (that is a factor). This should be an unquoted column name.
estimate	A vector of column identifiers, or one of dplyr selector functions to choose which variables contains the class probabilities. It defaults to the prefix used by tidymodels (.pred_). The order of the identifiers will be considered the same as the order of the levels of the truth variable.
metrics	A set of metrics passed created via yardstick::metric_set()
save_pred	Indicates whether to a column of post-calibration predictions.
...	Options to pass to cal_estimate_logistic(), such as the smooth argument.

Performance Metrics

By default, the average of the root mean square error (RMSE) is returned. Any appropriate yardstick::metric_set() can be used. The validation function compares the average of the metrics before, and after the calibration.

See Also

https://www.tidymodels.org/learn/models/calibration/, cal_estimate_linear()

Examples

library(dplyr)
library(yardstick)
library(rsample)

head(boosting_predictions_test)

reg_stats <- metric_set(rmse, ccc)

set.seed(828)
boosting_predictions_oob %>%
  # Resample with 10-fold cross-validation
  vfold_cv() %>%
  cal_validate_linear(truth = outcome, smooth = FALSE, metrics = reg_stats)

topepo/probably documentation built on Oct. 21, 2024, 3:28 a.m.