bride: Brier Score Decomposition

Description Usage Arguments Details Value Author(s) References Examples

View source: R/bride.R

Description

Decomposes the empirical Brier score into reliability, resolution and uncertainty. The components are computed in two different ways: Using the original (biased) estimators by Murphy (1974), and using bias-corrected estimators proposed by Ferro and Fricker (2012). Variance estimators are calculated. Can deal with probabilistic predictions of binary events.

Usage

1
  bride(p, y, n.bins=10, ...)

Arguments

p

a vector of forecast probabilities. No default.

y

a vector of binary event indicators. No default.

n.bins

number of bins used for calculating the calibration function. Default n.bins = 10.

...

possible additional arguments. Not used at the moment.

Details

The values of the forecast probabilities in p are binned into n.bins equidistant bins on the unit interval. Based on this binning, the calibration function

P(y=1|p)

is estimated, which is required to calculate the reliability and resolution term.

Value

An object of class bride, essentially a list containing:

p,y

the objects of the original request.

n.bins

number of equidistant, exhaustive bins used to calculate cross table.

p.binavgs

in-bin-average forecast probabilities.

p.binned

a vector of length(p), where each original p was replaced by its in-bin-average.

c.table

the cross-table, aka consistency table; c.table[i,j] indicates number of times that y was in the i-th class and at the same time p was in the j-th bin.

rel,res,unc

reliability, resolution, uncertainty estimates derived by Murphy (1970).

rel2,res2,unc2

bias-corrected reliability, resolution, uncertainty estimates derived by Ferro (2012).

relv,resv,uncv,rel2v,res2v,unc2v

variance estimators derived by Siegert (2012).

Author(s)

Stefan Siegert

References

Murphy, AH (1974) A new vector partition of the probability score, Journal of Applied Meteorology, 12:595-600

Ferro CAT, Fricker TE (2012) A bias-corrected decomposition of the Brier Score, Quarterly Journal of the Royal Meteorological Society, 138(668): 1954-1960

Siegert, S. (2013) Variance estimators for Brier Score decomposition, document in preparation

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
## number of forecasts and verifications
N <- 100
## produce some p-values
p <- runif(n=N)
## produce events with corresponding probabilities
y <- rbinom(n=N, size=1, prob=p)
## perform brier score decomposition
b <- bride(p=p,y=y)
## print
print(b)

Example output

Call:
bride.default(p = p, y = y)

REL = 0.0032 +/- 0.0094
RES = 0.1029 +/- 0.0232
UNC = 0.2500 +/- 0.0030

REL - RES + UNC = 0.1503
Br = 0.1493

bride documentation built on May 2, 2019, 5:15 p.m.