prauc_table.CV.SuperLearner: Table of cross-validated PR-AUCs from CV.SuperLearner result

In ck37/ck37r: Chris Kennedy's R Toolkit

Table of cross-validated PR-AUCs from CV.SuperLearner result

Description

Calculates cross-validated PR-AUC for each learner in the CV.SuperLearner. Also calculates standard-error, confidence interval and p-value. Based on initial code by Alan Hubbard.

Usage

## S3 method for class 'CV.SuperLearner'
prauc_table(
  x,
  y = x$Y,
  sort = TRUE,
  null_hypothesis = NULL,
  two_tailed = FALSE,
  lower.tail = TRUE,
  ...
)

Arguments

x	CV.SuperLearner object
y	Outcome vector, if not already added to CV.SL object.
sort	Sort table by order of AUC.
null_hypothesis	Not implemented yet
two_tailed	Not implemented yet
lower.tail	Not implemented yet
...	Any additional unused arguments, due to the prauc_table generic.

Value

Dataframe table with PR-AUC and std dev.

References

Boyd, K., Eng, K. H., & Page, C. D. (2013). Area under the precision-recall curve: point estimates and confidence intervals. In Joint European conference on machine learning and knowledge discovery in databases (pp. 451-466). Springer, Berlin, Heidelberg.

Polley EC, van der Laan MJ (2010) Super Learner in Prediction. U.C. Berkeley Division of Biostatistics Working Paper Series. Paper 226. http://biostats.bepress.com/ucbbiostat/paper266/

van der Laan, M. J., Polley, E. C. and Hubbard, A. E. (2007) Super Learner. Statistical Applications of Genetics and Molecular Biology, 6, article 25. http://www.degruyter.com/view/j/sagmb.2007.6.issue-1/sagmb.2007.6.1.1309/sagmb.2007.6.1.1309.xml

See Also

cvsl_auc plot_roc.SuperLearner ci.cvAUC

Examples

library(SuperLearner)
library(ck37r)

data(Boston, package = "MASS")

set.seed(1)
y = as.numeric(Boston$medv > 23)
cvsl = CV.SuperLearner(Y = y,
                       X = subset(Boston, select = -medv),
                       family = binomial(),
                       cvControl = list(V = 2, stratifyCV = TRUE),
                       SL.library = c("SL.mean", "SL.glm"))
prauc_table(cvsl, y = y)

ck37/ck37r documentation built on April 29, 2023, 11:42 p.m.