boot.paired.roc: Bootstrap paired ROC curves

In erikpeter/fbroc: Fast Algorithms to Bootstrap Receiver Operating Characteristics Curves

Description

Given two numerical predictors for the same outcome on the same set of samples, this functions enables the bootstrapping of the paired ROC curves of the two prediction models. While bootstrapping the same set of samples are used for both curves in each iteration, preserving the correlation between the two models.

Usage

boot.paired.roc(pred1, pred2, true.class, stratify = TRUE,
  n.boot = 1000, use.cache = FALSE, tie.strategy = NULL)

Arguments

pred1	Numerical predictions for the first classifier.
pred2	Numerical predictions for the second classifier.
true.class	A logical vector. TRUE indicates the sample belonging to the positive class.
stratify	Logical. Indicates whether we use stratified bootstrap. Default to TRUE. Non-stratified bootstrap is not yet implemented.
n.boot	A number that will be coerced to integer. Specified the number of bootstrap replicates. Defaults to 1000.
use.cache	If true the bootstrapping results for the ROC curve will be pre-cached. This increases speed when the object is used often, but also takes up more memory.
tie.strategy	How to handle ties. See details below.

Value

A list of class fbroc.paired.roc, containing the elements:

prediction1	Input predictions for first model.
prediction2	Input predictions for second model.
true.class	Input classes.
n.thresholds1	Number of thresholds of the first predictor.
n.thresholds2	Number of thresholds of the second predictor.
n.boot	Number of bootstrap replicates.
use.cache	Indicates if cache is used for this ROC object.
tie.strategy	Used setting how to handle ties in predictors.
n.pos	Number of positive observations.
n.neg	Number of negative observations.
roc1	A data.frame containing the thresholds of the first ROC curve and the TPR and FPR at these thresholds.
roc2	A data.frame containing the thresholds of the second ROC curve and the TPR and FPR at these thresholds.
auc1	The AUC of the first ROC curve.
auc2	The AUC of the second ROC curve.
boot.tpr1	If the cache is enabled, a matrix containing the bootstrapped TPR at the thresholds for the first predictor.
boot.fpr1	If the cache is enabled, a matrix containing the bootstrapped FPR at the thresholds for the first predictor.
boot.tpr2	If the cache is enabled, a matrix containing the bootstrapped TPR at the thresholds for the second predictor.
boot.fpr2	If the cache is enabled, a matrix containing the bootstrapped FPR at the thresholds for the second predictor.

Caching

If you enable caching, boot.roc calculates the requested number of bootstrap samples and saves the TPR and FPR values for each iteration. This can take up a sizable portion of memory, but it speeds up subsequent operations. This can be useful if you plan to use the ROC curve multiple fbroc functions.

Ties

You can set this parameter to either 1 or 2. If your numerical predictor has no ties, both settings will produce the same results. If you set tie.strategy to 1 the ROC curve is built by connecting the TPR/FPR pairs for neighboring thresholds. A tie.strategy of 2 indicates that the TPR calculated at a specific FPR is the best TPR at a FPR smaller than or equal than the FPR specified. Defaults to 2.

See Also

boot.roc, plot.fbroc.paired.roc, perf.fbroc.paired.roc

Examples

data(roc.examples)
# Do not use cache
example <- boot.paired.roc(roc.examples$Cont.Pred, roc.examples$Cont.Pred.Outlier,
                          roc.examples$True.Class, n.boot = 500)
perf(example, "auc") # estimate difference in auc
perf(example, "tpr", fpr = 0.5) # estimate difference in TPR at a FPR of 50%
plot(example) # show plot
# Cached mode
example <- boot.paired.roc(roc.examples$Cont.Pred, roc.examples$Cont.Pred.Outlier,
                          roc.examples$True.Class, n.boot = 1000, use.cache = TRUE)
conf(example, conf.for = "tpr", steps = 10) # get confidence regions for TPR at FPR
conf(example, conf.for = "fpr", steps = 10) # get confidence regions for FPR at TPR
perf(example, "fpr", tpr = 0.9) # estimate difference in FPR at a TPR of 90%                     

erikpeter/fbroc documentation built on May 16, 2019, 8:42 a.m.