mtr_brier_score: Brier score

In chuvanan/metrics: Metrics For Evaluating Machine Learning Models

Description

Brier score measures the accuracy of probabilistic predictions; the smaller the Brier score, the better. The Brier score always takes on a value between zero and one.

Given N predictions, the Brier score is defined by the mean squared difference between (1) the estimated probability assigned to the possible outcome and (2) the actual outcome.

Here is a few examples that help the interpretation of Brier score.

Suppose that one is forecasting the probability P that it will rain on a given day. Then the Brier score is calculated as follows:

• If the forecast is 100% (P = 1) and it rains, then the Brier Score is 0, the best score achievable.

• If the forecast is 100% and it does not rain, then the Brier Score is 1, the worst score achievable.

• If the forecast is 70% (P = 0.70) and it rains, then the Brier Score is (0.70 − 1)^2 = 0.09.

• If the forecast is 30% (P = 0.30) and it rains, then the Brier Score is (0.30 − 1)^2 = 0.49.

Usage

mtr_brier_score(actual, predicted)

Arguments

actual	[numeric] Ground truth binary numeric vector containing 1 for the positive class and 0 for the negative class.
predicted	[numeric] A vector of estimated probabilities.

Value

A scalar numeric output

Author(s)

An Chu

See Also

mtr_log_loss mtr_mean_log_loss

Examples

act <- c(0, 1, 1, 0)
pred <- c(0.1, 0.9, 0.8, 0.3)
mtr_brier_score(act, pred)

chuvanan/metrics documentation built on Nov. 4, 2019, 8:52 a.m.