bbrier: Binary Brier Score

In mlr3measures: Performance Measures for 'mlr3'

Binary Brier Score

Description

Measure to compare true observed labels with predicted probabilities in binary classification tasks.

Usage

bbrier(truth, prob, positive, sample_weights = NULL, ...)

Arguments

truth	(factor()) True (observed) labels. Must have the exactly same two levels and the same length as response.
prob	(numeric()) Predicted probability for positive class. Must have exactly same length as truth.
positive	(⁠character(1))⁠ Name of the positive class.
sample_weights	(numeric()) Vector of non-negative and finite sample weights. Must have the same length as truth. The vector gets automatically normalized to sum to one. Defaults to equal sample weights.
...	(any) Additional arguments. Currently ignored.

Details

The Binary Brier Score is defined as

\frac{1}{n} \sum_{i=1}^n w_i (I_i - p_i)^2,

where w_i are the sample weights, and I_{i} is 1 if observation x_i belongs to the positive class, and 0 otherwise.

Note that this (more common) definition of the Brier score is equivalent to the original definition of the multi-class Brier score (see mbrier()) divided by 2.

Value

Performance value as numeric(1).

Meta Information

• Type: "binary"

• Range: [0, 1]

• Minimize: TRUE

• Required prediction: prob

References

https://en.wikipedia.org/wiki/Brier_score

Brier GW (1950). “Verification of forecasts expressed in terms of probability.” Monthly Weather Review, 78(1), 1–3. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2")}.

See Also

Other Binary Classification Measures: auc(), dor(), fbeta(), fdr(), fn(), fnr(), fomr(), fp(), fpr(), gmean(), gpr(), npv(), ppv(), prauc(), tn(), tnr(), tp(), tpr()

Examples

set.seed(1)
lvls = c("a", "b")
truth = factor(sample(lvls, 10, replace = TRUE), levels = lvls)
prob = runif(10)
bbrier(truth, prob, positive = "a")

mlr3measures documentation built on Sept. 12, 2024, 7:20 a.m.