make_perf_df: Take a list of performance metrics and return the observed...

In gweissman/gmish: Miscellaneous Functions for Predictive Modeling

Take a list of performance metrics and return the observed estimate and 95% confidence intervals based on 1000 bootstrap replicates in the form a data.frame.

Description

Take a list of performance metrics and return the observed estimate and 95% confidence intervals based on 1000 bootstrap replicates in the form a data.frame.

Usage

make_perf_df(
  preds,
  obs,
  metrics = list("brier", "sbrier", "ici", "lloss", "cstat"),
  ...
)

Arguments

preds	A vector of predicted probabilities for the first model.
obs	A vector containing the observed binary outcomes (0 or 1).
metrics	A list of names of functions as characters that are of the form f(preds, obs), e.g. "cstat"
...	Additional arguments for the particular metric or boot function, e.g. 'thresh = 0.6'

Examples

# Generate some predictions for two different models
preds <- runif(1000)
# Generate some binary outcomes
obs <- sample(0:1, size = 1000, replace = TRUE)
# Calculate the Confidence interval around the estimate of the Brier Score
make_perf_df(preds, obs, metrics = list('brier', 'cstat'))

gweissman/gmish documentation built on Feb. 21, 2025, 1:20 a.m.