bs_ci: Calculate the bias-corrected, basic bootstrapped confidence...

In gweissman/gmish: Miscellaneous Functions for Predictive Modeling

Calculate the bias-corrected, basic bootstrapped confidence interval for a given performance metric.

Description

Calculate the bias-corrected, basic bootstrapped confidence interval for a given performance metric.

Usage

bs_ci(
  preds,
  obs = NULL,
  metric = NULL,
  reps = 1000,
  conf = 0.95,
  seed = NULL,
  btype = "basic",
  use_parallel = "multicore",
  ...
)

Arguments

preds	A vector of predicted probabilities.
obs	A vector containing the observed binary outcomes (0 or 1).
metric	A function of the form f(preds, obs), e.g. 'bs'
reps	The number of bootstrap replicates. Default = 1000.
conf	The width of the confidence interval. Default = 0.95.
seed	An optional random seed.
btype	The type of bootstrap to calculate. Default is 'basic', also takes 'bca', or other types supported in the 'boot' package.
use_parallel	Provides the parallel backend to the boot package. Default is 'multicore'. Use 'no' to turn off parallel processing.
...	Additional arguments for the particular metric function, e.g. 'thresh = 0.6'

Examples

# Generate some predictions
predictions <- runif(1000)
# Generate some binary outcomes
observations <- sample(0:1, size = 1000, replace = TRUE)
# Calculate the Confidence interval around the estimate of the Brier Score
bs_ci(predictions, observations, metric = brier)

gweissman/gmish documentation built on Feb. 21, 2025, 1:20 a.m.