bride: Brier Score Decomposition

In bride: Brier score decomposition of probabilistic forecasts for binary events

Description

Decomposes the empirical Brier score into reliability, resolution and uncertainty. The components are computed in two different ways: Using the original (biased) estimators by Murphy (1974), and using bias-corrected estimators proposed by Ferro and Fricker (2012). Variance estimators are calculated. Can deal with probabilistic predictions of binary events.

Usage

  bride(p, y, n.bins=10, ...)

Arguments

p	a vector of forecast probabilities. No default.
y	a vector of binary event indicators. No default.
n.bins	number of bins used for calculating the calibration function. Default n.bins = 10.
...	possible additional arguments. Not used at the moment.

Details

The values of the forecast probabilities in p are binned into n.bins equidistant bins on the unit interval. Based on this binning, the calibration function

P(y=1|p)

is estimated, which is required to calculate the reliability and resolution term.

Value

An object of class bride, essentially a list containing:

p,y	the objects of the original request.
n.bins	number of equidistant, exhaustive bins used to calculate cross table.
p.binavgs	in-bin-average forecast probabilities.
p.binned	a vector of length(p), where each original p was replaced by its in-bin-average.
c.table	the cross-table, aka consistency table; c.table[i,j] indicates number of times that y was in the i-th class and at the same time p was in the j-th bin.
rel,res,unc	reliability, resolution, uncertainty estimates derived by Murphy (1970).
rel2,res2,unc2	bias-corrected reliability, resolution, uncertainty estimates derived by Ferro (2012).
relv,resv,uncv,rel2v,res2v,unc2v	variance estimators derived by Siegert (2012).

Author(s)

Stefan Siegert

References

Murphy, AH (1974) A new vector partition of the probability score, Journal of Applied Meteorology, 12:595-600

Ferro CAT, Fricker TE (2012) A bias-corrected decomposition of the Brier Score, Quarterly Journal of the Royal Meteorological Society, 138(668): 1954-1960

Siegert, S. (2013) Variance estimators for Brier Score decomposition, document in preparation

Examples

## number of forecasts and verifications
N <- 100
## produce some p-values
p <- runif(n=N)
## produce events with corresponding probabilities
y <- rbinom(n=N, size=1, prob=p)
## perform brier score decomposition
b <- bride(p=p,y=y)
## print
print(b)

Example output

Call:
bride.default(p = p, y = y)

REL = 0.0032 +/- 0.0094
RES = 0.1029 +/- 0.0232
UNC = 0.2500 +/- 0.0030

REL - RES + UNC = 0.1503
Br = 0.1493

bride documentation built on May 2, 2019, 5:15 p.m.