bias_sample: Determine bias of forecasts

In scoringutils: Utilities for Scoring and Assessing Predictions

Determine bias of forecasts

Description

Determines bias from predictive Monte-Carlo samples. The function automatically recognises whether forecasts are continuous or integer valued and adapts the Bias function accordingly.

Usage

bias_sample(observed, predicted)

Arguments

observed	A vector with observed values of size n
predicted	nxN matrix of predictive samples, n (number of rows) being the number of data points and N (number of columns) the number of Monte Carlo samples. Alternatively, if n = 1, predicted can just be a vector of size n.

Details

For continuous forecasts, Bias is measured as

B_t (P_t, x_t) = 1 - 2 * (P_t (x_t))

where P_t is the empirical cumulative distribution function of the prediction for the observed value x_t. To handle ties appropriately (which can occur when predictions equal observations for exampele due to rounding), P_t(x_t) is computed using mid-ranks: the fraction of predictive samples strictly smaller than x_t plus half the fraction equal to x_t.

For integer valued forecasts, Bias is measured as

B_t (P_t, x_t) = 1 - (P_t (x_t) + P_t (x_t + 1))

to adjust for the integer nature of the forecasts.

In both cases, Bias can assume values between -1 and 1 and is 0 ideally.

Value

Numeric vector of length n with the biases of the predictive samples with respect to the observed values.

Input format

Overview of required input format for sample-based forecasts

References

The integer valued Bias function is discussed in Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15 Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al. (2019) Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15. PLOS Computational Biology 15(2): e1006785. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1371/journal.pcbi.1006785")}

Examples

## integer valued forecasts
observed <- rpois(30, lambda = 1:30)
predicted <- replicate(200, rpois(n = 30, lambda = 1:30))
bias_sample(observed, predicted)

## continuous forecasts
observed <- rnorm(30, mean = 1:30)
predicted <- replicate(200, rnorm(30, mean = 1:30))
bias_sample(observed, predicted)

scoringutils documentation built on April 6, 2026, 1:07 a.m.