setup_CVPlus: Setup CV+ Conformal Prediction

In rtemis: Schema-Driven Machine Learning and Visualization

Setup CV+ Conformal Prediction

Description

Configure CV+, jackknife+ and cross-conformal prediction for conformal over a model trained with outer resampling.

Usage

setup_CVPlus(alpha = 0.1, score = NULL, seed = NULL)

Arguments

alpha	Numeric (0, 1): Miscoverage rate. The region covers with probability at least 1 - 2 * alpha in the worst case.
score	Optional Character {"absolute", "LAC", "APS"}: Nonconformity score. NULL resolves to "absolute" for a regression and "APS" for a classification. See setup_SplitConformal for the trade between the two set-valued scores, and for why APS draws.
seed	Optional Integer [0, Inf): Random seed for the APS draw. NULL draws one and records it.

Details

CV+ (Barber, Candes, Ramdas and Tibshirani, 2021) spends no data on calibration: every case is out-of-fold exactly once, so every case both trains and calibrates. Over a SupervisedRes the expensive part is already paid – the per-fold models and their out-of-fold predictions are stored – and what remains is predicting newdata under each fold's model and taking two order statistics.

It delivers 1 - 2 * alpha in the worst case, not 1 - alpha. The bound is conservative and observed coverage usually sits near 1 - alpha; the region states the guarantee it carries.

The folds must partition the cases, each case out-of-fold exactly once. setup_KFold() and setup_LOOCV() do; the subsampling and bootstrap types do not, and conformal() refuses them by checking the indices.

Which relative runs follows from the object: leave-one-out folds make it jackknife+, and a classification outcome makes it cross-conformal (Vovk, 2015), CV+ being defined for intervals. The result records which.

Value

CVPlusConfig object.

Author(s)

EDG

Examples

setup_CVPlus()
setup_CVPlus(alpha = 0.2)

rtemis documentation built on Oct. 20, 2026, 1:06 a.m.