auc_ci: Calculate CIs of ROC and precision-recall AUCs

Description Usage Arguments Value See Also Examples

View source: R/g_auc_ci.R

Description

The auc_ci function takes an S3 object generated by evalmod and calculates CIs of AUCs when multiple data sets are specified.

Usage

1
2
3
4
auc_ci(curves, alpha = NULL, dtype = NULL)

## S3 method for class 'aucs'
auc_ci(curves, alpha = 0.05, dtype = "normal")

Arguments

curves

An S3 object generated by evalmod. The auc_ci function accepts the following S3 objects.

S3 object # of models # of test datasets
smcurves single multiple
mmcurves multiple multiple

See the Value section of evalmod for more details.

alpha

A numeric value of the significant level (default: 0.05)

dtype

A string to specify the distribution used for CI calculation.

dtype distribution
normal (default) Normal distribution
z Normal distribution
t t-distribution

Value

The auc_ci function returns a dataframe of AUC CIs.

See Also

evalmod for generating S3 objects with performance evaluation measures. auc for retrieving a dataset of AUCs.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
##################################################
### Single model & multiple test datasets
###

## Create sample datasets with 100 positives and 100 negatives
samps <- create_sim_samples(4, 100, 100, "good_er")
mdat <- mmdata(samps[["scores"]], samps[["labels"]],
               modnames = samps[["modnames"]],
               dsids = samps[["dsids"]])

## Generate an smcurve object that contains ROC and Precision-Recall curves
smcurves <- evalmod(mdat)

## Calculate CI of AUCs
sm_auc_cis <- auc_ci(smcurves)

## Shows the result
sm_auc_cis

##################################################
### Multiple models & multiple test datasets
###

## Create sample datasets with 100 positives and 100 negatives
samps <- create_sim_samples(4, 100, 100, "all")
mdat <- mmdata(samps[["scores"]], samps[["labels"]],
               modnames = samps[["modnames"]],
               dsids = samps[["dsids"]])

## Generate an mscurve object that contains ROC and Precision-Recall curves
mmcurves <- evalmod(mdat)

## Calculate CI of AUCs
mm_auc_ci <- auc_ci(mmcurves)

## Shows the result
mm_auc_ci

precrec documentation built on Feb. 2, 2021, 5:14 p.m.