bride: Brier Score Decomposition

In bride: Brier score decomposition of probabilistic forecasts for binary events

Description

Decomposes the empirical Brier score into reliability, resolution and uncertainty. Two different estimators for the components are provided: The original estimators proposed by Murphy (1974), and the bias-corrected estimators proposed by Ferro and Fricker (2012). Sampling variances of all the components are estimated. This package applies only to probabilistic predictions of binary events.

Usage

  bride(p, y, n.bins=10, ...)

Arguments

p	a vector of forecast probabilities. No default.
y	a vector of binary event indicators. No default.
n.bins	number of bins used for calculating the calibration function. Default n.bins = 10.
...	possible additional arguments. Not used at the moment.

Details

The values of the forecast probabilities in p are binned into n.bins bins of equal length on the unit interval. All probabilities are replaced with their in-bin average. Based on this binning, the calibration function P(y=1|p) is estimated, which is required to estimate the components of the Brier Score decomposition of p.

Value

An object of class bride, essentially a list containing:

p,y	the objects of the original request.
n.bins	number of equidistant, exhaustive bins used to calculate the cross table.
rel,res,unc	reliability, resolution, uncertainty estimates derived by Murphy (1970).
rel2,res2,unc2	bias-corrected reliability, resolution, uncertainty estimates derived by Ferro and Fricker (2012).
rel.var,res.var,unc.var,rel2.var,res2.var,unc2.var	variance estimators derived by Siegert (2013).

Author(s)

Stefan Siegert

References

Murphy, AH (1974) A new vector partition of the probability score, Journal of Applied Meteorology, 12:595-600

Ferro CAT, Fricker TE (2012) A bias-corrected decomposition of the Brier Score, Quarterly Journal of the Royal Meteorological Society, 138(668): 1954-1960

Siegert, S. (2013) Variance estimation for Brier Score decomposition, http://arxiv.org/abs/1303.6182

Examples

## number of forecasts and verifications
N <- 100
## produce some forecasts
p <- runif(n=N)
## produce events y[i] that occur with probability p[i]
y <- rbinom(n=N, size=1, prob=p)
## perform brier score decomposition
b <- bride(p=p,y=y)
## print
print(b)

bride documentation built on May 29, 2017, 5:41 p.m.