p.boot: Bootstrap confidence interfals of classifier performances

View source: R/MPMutils.R

p.bootR Documentation

Bootstrap confidence interfals of classifier performances

Description

Compute bootstrap confidence intervals for various classifier performance indices.

Usage

p.boot(x, rep = 5000, ci.method = "bca", formula = "f1", ...)

Arguments

x

An data.frame of n rows (subjects) and 2 columns: the first column contains the predicted values for an outcome y (i.e., y.hat), and the second column contains the observed ("true") values for y.

rep

Number of bootstrap iterations (default = 5000). A high number of iterations is needed for reliable estimations. If rep < 5000, estimation errors might occur.

ci.method

Method used for bootstrap confidence interval estimation (default = "bca"; i.e., adjusted bootstrap percentile).

formula

Performance index. One among: "f1" (default), "accuracy", "sensitivity", "specificity", "ppv" (or "precision"; i.e., positive predictive value), "npv" (negative predictive value), "plr" (positive likelihood ratio), "nlr" (hegative likelihood ratio), "fpr" (false positive rate), "fdr" (false discovery rate), "fnr" (flase negative rate), "fnc" (false negative cost = FN/(TP + TN), see Fragomeni et al. 2022).

...

Currently ignored.

Value

A list of 2 objects:

  1. "boot", an object of class boot, containing bootstrap replicates of the given statistic;

  2. "ci", an object of class bootci, containing bootstrap confidence interval estimations.

Author(s)

Fernando Palluzzi fernando.palluzzi@gmail.com

References

Fragomeni SM, Moro F, Palluzzi F, Mascilini F, Rufini V, Collarino A, Inzani F, Giacò L, Scambia G, Testa AC, Garganese G (2022). Evaluating the risk of inguinal lymph node metastases before surgery using the Morphonode Predictive Model: a prospective diagnostic study. Ultrasound xx Xxxxxxxxxx xxx Xxxxxxxxxx. 00(0):000-000. <https://doi.org/00.0000/00000000000000000000>

Davison AC and Hinkley DV (1997). Bootstrap Methods and Their Application, Chapter 5. Cambridge University Press.

DiCiccio TJ and Efron B (1996). Bootstrap confidence intervals (with Discussion)._Statistical Science_, *11*, 189-228.

Efron B (1987). Better bootstrap confidence intervals (with Discussion). Journal of the American Statistical Association, *82*, 171-200. <https://doi.org/10.2307/2289144>

See Also

See boot.ci for further details.

Examples



# Predicted ultrasound phenotype from the default morphonode RFC1
y.hat <- predict(mpm.rfc$rfc$RFC1, mpm.rfc$validation$V1)

# Actual ultrasound phenotype values
y <- mpm.rfc$validation$V1$y

# Input preparation
Y <- data.frame(y.hat, y)

# F1 score bootstrap confidence intervals
F1 <- p.boot(Y)
print(F1$boot$t0)              # F1 score observed value
print(F1$ci$bca[4:5])          # F1 score bca confidence interval

# Accuracy bootstrap confidence intervals
A <- p.boot(Y, formula = "accuracy")
print(A$boot$t0)               # Accuracy observed value
print(A$ci$bca[4:5])           # Accuracy bca confidence interval




Morphonodepredictivemodel/morphonode documentation built on Feb. 15, 2023, 4:51 a.m.