get_brier: Calculate Brier score and weighted class probability average.

In PiotrTymoszuk/caretExtra: Extra functionality for predictions and quality control of caret models

Calculate Brier score and weighted class probability average.

Description

get_brier() computes the Brier score (statistic name brier_score). get_classp() computes a weighted arithmetic mean of the winning class assignment probabilities. The weights are frequencies of the true outcome classes(statistic name class_p_outcome) and predicted classes (class_p_fitted). For cross-validation, mean metrics with 95% confidence intervals are returned.

Usage

get_brier(data, ci_method = c("percentile", "bca", "norm"))

get_classp(data, ci_method = c("percentile", "bca", "norm"))

Arguments

data	a data frame with the columns .outcome, .fitted, and, optionally .resample. The column square_dist is required for the Brier scores. The column winner_p is required for the average probabilites.
ci_method	method used for 95% CI calculation.

Value

a tibble with the columns statistic, estimate, lower_ci and upper_ci storing the statistic name, its estimated value as well as lower and upper limit of the 95% confidence interval.

PiotrTymoszuk/caretExtra documentation built on Oct. 15, 2023, 10:03 p.m.