compute_calibration: Estimate calibration given forecasts and corresponding...

In ghurault/HuraultMisc: Guillem Hurault Functions' Library

Estimate calibration given forecasts and corresponding outcomes

Description

Estimate calibration given forecasts and corresponding outcomes

Usage

compute_calibration(
  forecast,
  outcome,
  method = c("smoothing", "binning"),
  CI = NULL,
  binwidth = NULL,
  ...
)

Arguments

forecast	Vector of probability forecasts.
outcome	Vector of observations (0 or 1).
method	Method used to estimate calibration, either "smoothing" or "binning".
CI	Confidence level (e.g. 0.95). CI not computed if NULL (CI can be expensive to compute for LOWESS).
binwidth	Binwidth when calibration is estimated by binning. If NULL, automatic bin width selection with Sturges' method.
...	Arguments of stats::loess() function (e.g. span).

Value

Dataframe with columns Forecast (bins), Frequency (frequency of outcomes in the bin), Lower (lower bound of the CI) and Upper (upper bound of the CI).

Examples

N <- 1e4
f <- rbeta(N, 1, 1)
o <- sapply(f, function(x) {rbinom(1, 1, x)})
lapply(c("binning", "smoothing"),
       function(m) {
         cal <- compute_calibration(f, o, method = m)
         with(cal, plot(Forecast, Frequency, type = "l"))
         abline(c(0, 1), col = "red")
       })

ghurault/HuraultMisc documentation built on Aug. 1, 2022, 3:11 p.m.