train.logreg: Train calibration of probabilistic classifier

In davidavdav/ROC: Compute structures to compute ROC and DET plots and metrics for 2-class classifiers.

Train calibration of probabilistic classifier

Description

These functions train a transform of scores such that these can be interpreted better as a log-likelihood-ratio. The resulting scores should have a lower Cllr.

Usage

 cal <- train.logreg(x, dep="score")
 cal <- train.cmlg(x)

Arguments

x	A data frame of class cst or something that can be coerced into that
dep	A string indiceting the formula the target depends on for model fitting

Details

train.logreg performs logistic regression by fitting the generalized linear model target ~ score. The weights of the trials are chosen inversely proportional to the class priors, so that effectively the model produces a log likelihood ratio instead of log posterior odds.

train.cmlg is a light-weight linear calibration function that assumes equal-variance Gaussian distribution of the target and non-target scores.

Value

A model, or a function of class linearcalibration, such that the function predict can be applied to the calibration.

Author(s)

David A. van Leeuwen

Examples

## These data were already calibrated
data(tno.2008)
summary(roc(tno.2008))
## now re-calibrate with a fairly complicated model
train <- subset(tno.2008, as.character(model)<65357)
tst <- subset(tno.2008, as.character(model)>=65357)
cal <- train.logreg(train, "score + mmic + tmic")
cal 
x <- transform(tst, score=predict(cal, tst))
det.plot(x)
## re-calibrate with a simple equal-variance Gaussian model
cal <- train.cmlg(train)
x <- transform(tst, score=predict(cal, tst))
summary(roc(x))

davidavdav/ROC documentation built on Sept. 8, 2023, 2:39 p.m.