compute_interval_coverage: Empirical prediction-interval coverage

In forecastsuite: Multi-Model Time Series Forecasting Toolkit and Local App

Empirical prediction-interval coverage

Description

Whether a model's uncertainty bounds are honest, not just its point forecast: what fraction of actuals actually fell inside yhat_lower/yhat_upper, versus the interval's nominal confidence level (Prophet and the forecast-package adapters default to an 80% interval).

Usage

compute_interval_coverage(fc_tib, test_df, nominal_level = 0.8)

Arguments

fc_tib	A tibble with ds/yhat columns, and typically yhat_lower/yhat_upper. Models with no interval columns (e.g. NNETAR without PI = TRUE) return NA coverage rather than erroring.
test_df	A tibble with ds/y columns – the held-out actuals.
nominal_level	Numeric, the interval's nominal confidence level. Default 0.8.

Value

A list with empirical_coverage, nominal_level, gap (empirical_coverage - nominal_level), and n.

Examples

ds <- as.Date("2024-01-01") + 0:9
fc_tib <- tibble::tibble(ds = ds, yhat = 10, yhat_lower = 9, yhat_upper = 11)
test_df <- tibble::tibble(ds = ds, y = c(rep(10, 8), 20, 20))
compute_interval_coverage(fc_tib, test_df)

forecastsuite documentation built on Oct. 18, 2026, 5:08 p.m.