setup_SplitConformal: Setup Split Conformal Prediction

In rtemis: Schema-Driven Machine Learning and Visualization

Setup Split Conformal Prediction

Description

Configure split conformal prediction for conformal.

Usage

setup_SplitConformal(alpha = 0.1, score = NULL, seed = NULL)

Arguments

alpha	Numeric (0, 1): Miscoverage rate. The region covers with probability at least 1 - alpha.
score	Optional Character {"absolute", "LAC", "APS"}: Nonconformity score. NULL resolves to "absolute" for a regression and "APS" for a classification.
seed	Optional Integer [0, Inf): Random seed for the APS draw. NULL draws one and records it.

Details

The reference construction: score every calibration case, take q to be the score at rank ceiling((n + 1) * (1 - alpha)) from the bottom, and return every outcome whose score would not exceed it. Coverage is at least 1 - alpha, and needs only that the calibration cases and the test case be exchangeable.

alpha has a floor set by the calibration count. A finite region needs ceiling((n + 1) * (1 - alpha)) <= n, so alpha = 0.05 needs at least 19 calibration cases and alpha = 0.01 at least 99. conformal() aborts naming the count.

score is the real choice, and only for a classification. A regression has one score, abs(y - yhat), which is what NULL resolves to.

• "APS", the default, accumulates the predicted probabilities from the most probable class down to the true one. Larger sets, markedly better coverage on the hard cases – which is the reason to want a set at all.

• "LAC" scores 1 - p(y). It produces the smallest sets attaining valid marginal coverage, and it buys them by under-covering exactly the cases where the model is unsure. Choose it when small sets are the point and the trade is understood.

APS is randomized (Romano, Sesia and Candes, 2020): the true label's own probability enters its score multiplied by a uniform draw, one per case. That is not a refinement. Without it, every case whose true label ranks last scores exactly 1, so a model erring on more than alpha of its cases puts the threshold at 1 and every set holds every label – valid coverage carrying no information. seed keeps the draw reproducible: left NULL, one is drawn and recorded on the returned config, so an unseeded region can still be reproduced from what it reports. "LAC" draws nothing and ignores it.

Value

SplitConformalConfig object.

Author(s)

EDG

Examples

setup_SplitConformal()
setup_SplitConformal(alpha = 0.05)
setup_SplitConformal(score = "LAC")

rtemis documentation built on Oct. 20, 2026, 1:06 a.m.