compute_calibration: Estimate calibration given forecasts and corresponding...

Description Usage Arguments Value Examples

View source: R/calibration.R

Description

Estimate calibration given forecasts and corresponding outcomes

Usage

1
2
3
4
5
6
7
8
compute_calibration(
  forecast,
  outcome,
  method = c("smoothing", "binning"),
  CI = NULL,
  binwidth = NULL,
  ...
)

Arguments

forecast

Vector of probability forecasts.

outcome

Vector of observations (0 or 1).

method

Method used to estimate calibration, either "smoothing" or "binning".

CI

Confidence level (e.g. 0.95). CI not computed if NULL (CI can be expensive to compute for LOWESS).

binwidth

Binwidth when calibration is estimated by binning. If NULL, automatic bin width selection with 'Sturges' method.

...

Arguments of stats::loess() function (e.g. span)

Value

Dataframe with columns Forecast (bins), Frequency (frequency of outcomes in the bin), Lower (lower bound of the CI) and Upper (upper bound of the CI).

Examples

1
2
3
4
5
6
7
8
9
N <- 1e4
f <- rbeta(N, 1, 1)
o <- sapply(f, function(x) {rbinom(1, 1, x)})
lapply(c("binning", "smoothing"),
       function(m) {
         cal <- compute_calibration(f, o, method = m)
         with(cal, plot(Forecast, Frequency, type = "l"))
         abline(c(0, 1), col = "red")
       })

HuraultMisc documentation built on Sept. 6, 2021, 9:09 a.m.