calibrate_cv: Calibrate cross-validated model

In egenn/rtemis: Machine Learning and Visualization

Calibrate cross-validated model

Description

Calibrate cross-validated model trained using train_cv

Usage

calibrate_cv(
  mod,
  alg = "isotonic",
  learn.params = list(),
  resample.params = setup.resample(resampler = "kfold", n.resamples = 5, seed = NULL),
  which.repeat = 1,
  verbosity = 1,
  debug = FALSE
)

Arguments

mod	rtModCV object returned by train_cv
alg	Character: Algorithm to use for calibration
learn.params	List: List of parameters to pass to the learning algorithm
resample.params	List of parameters to pass to the resampling algorithm. Build using setup.resample
which.repeat	Integer: Which repeat to use for calibration
verbosity	Integer: 0: silent, > 0: print messages
debug	Logical: If TRUE, run without parallel processing, to allow better debugging.

Details

This is a work in progress to be potentially incorporated into train_cv You start by training a cross-validated model using train_cv, then this function can be used to calibrate the model. In order to use all available data, each outer resample from the input mod is resampled (using 5-fold CV by default) to train and test calibration models. This allows using the original label-based metrics of mod and also extract calibration metrics based on the same data, after aggregating the test set predictions of the calibration models.

Value

List: Calibrated models, test-set labels, test set performance metrics, estimated probabilities (uncalibrated), calibrated probabilities,

Author(s)

E.D. Gennatas

egenn/rtemis documentation built on Dec. 17, 2024, 6:16 p.m.