bias_range: Determines Bias of Quantile Forecasts

In scoringutils: Utilities for Scoring and Assessing Predictions

Determines Bias of Quantile Forecasts

Description

Determines bias from quantile forecasts. For an increasing number of quantiles this measure converges against the sample based bias version for integer and continuous forecasts.

Usage

bias_range(range, lower, upper, true_value)

Arguments

range	vector of corresponding size with information about the width of the central prediction interval
lower	vector of length corresponding to the number of central prediction intervals that holds predictions for the lower bounds of a prediction interval
upper	vector of length corresponding to the number of central prediction intervals that holds predictions for the upper bounds of a prediction interval
true_value	a single true value

Details

For quantile forecasts, bias is measured as

B_t = (1 - 2 * max(i | q_{t,i} in Q_t and q_{t,i} <= x_t\)) 1( x_t <= q_{t, 0.5}) + (1 - 2 * min(i | q_{t,i} in Q_t and q_{t,i} >= x_t)) 1( x_t >= q_{t, 0.5}),

where Q_t is the set of quantiles that form the predictive distribution at time t. They represent our belief about what the true value x_t will be. For consistency, we define Q_t such that it always includes the element q_{t, 0} = - ∞ and q_{t,1} = ∞. 1() is the indicator function that is 1 if the condition is satisfied and $0$ otherwise. In clearer terms, B_t is defined as the maximum percentile rank for which the corresponding quantile is still below the true value, if the true value is smaller than the median of the predictive distribution. If the true value is above the median of the predictive distribution, then $B_t$ is the minimum percentile rank for which the corresponding quantile is still larger than the true value. If the true value is exactly the median, both terms cancel out and B_t is zero. For a large enough number of quantiles, the percentile rank will equal the proportion of predictive samples below the observed true value, and this metric coincides with the one for continuous forecasts.

Bias can assume values between -1 and 1 and is 0 ideally.

Value

scalar with the quantile bias for a single quantile prediction

Author(s)

Nikos Bosse [e-mail]

Examples

lower <- c(
  6341.000, 6329.500, 6087.014, 5703.500,
  5451.000, 5340.500, 4821.996, 4709.000,
  4341.500, 4006.250, 1127.000, 705.500
)

upper <- c(
  6341.000, 6352.500, 6594.986, 6978.500,
  7231.000, 7341.500, 7860.004, 7973.000,
  8340.500, 8675.750, 11555.000, 11976.500
)

range <- c(0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95, 98)

true_value <- 8062

bias_range(
  lower = lower, upper = upper,
  range = range, true_value = true_value
)

scoringutils documentation built on Feb. 16, 2023, 7:30 p.m.