compute_zp: Compute ZP Quantile Loss

In RCtest: Reality Check and Predictive Ability Tests for Forecast Evaluation

Compute ZP Quantile Loss

Description

Computes the per-period ZP quantile loss matrix based on the squared difference between the indicator of a tail event and the forecast's predicted probability of that event (Corradi & Swanson, 2006, eq. 7).

Usage

compute_zp(
  forecast_matrix,
  forecast_sd_models,
  threshold,
  benchmark_col = ncol(forecast_matrix)
)

Arguments

forecast_matrix	matrix of dimension P x K_total. The benchmark column supplies the realized values y_t.
forecast_sd_models	matrix of dimension P x K, where K = K_total - 1. Contains time-varying forecast standard deviations, typically from estimate_forecast_variance.
threshold	numeric tail threshold \tau. The ZP loss measures how well each model predicts the probability of y_t \leq \tau. Typically set to a low quantile of the realized series, e.g., quantile(realized, 0.05) for the 5th-percentile left tail. In run_comprehensive_erc_analysis this is computed automatically as quantile(realizations, zp_quantile).
benchmark_col	Index or name of the benchmark column. Defaults to the last column.

Details

For each competing forecast k and period t:

ZP_{t,k} = \left(\mathbf{1}(y_t \leq \tau) - \Phi\!\left(\frac{\tau - \hat{y}_{t,k}}{\hat{\sigma}_{t,k}}\right)\right)^2

where y_t is the realized value, \tau is the threshold, \hat{y}_{t,k} is the point forecast, and \hat{\sigma}_{t,k} is the forecast standard deviation. Lower ZP values are better.

Choosing \tau at the 5th percentile focuses evaluation on whether forecasts correctly predict the risk of falling into the worst 5% of outcomes. The benchmark column is assigned a point-mass predictive distribution (\hat{\sigma} = 10^{-6}), which approximates the Brier score for the tail indicator and serves as a conservative reference. When the benchmark is the Historical Average (HA), the ZP test thus evaluates whether any competing forecast's calibrated tail probability outperforms the HA's point prediction of the tail event.

Value

matrix of dimension P x K_total containing ZP loss values. Lower values indicate better left-tail probability calibration. The benchmark column uses \hat{\sigma} = 10^{-6}.

References

Corradi, V., & Swanson, N. R. (2006). Predictive density and conditional confidence interval accuracy tests. Journal of Econometrics, 135(1–2), 187–228. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.jeconom.2005.07.026")}

Corradi, V., & Swanson, N. R. (2011). The White Reality Check and some of its recent extensions. In Festschrift in honor of Halbert L. White.

See Also

reality_check_zp_test, estimate_forecast_variance

Examples

data(metals)
benchmark_col      <- 15
K_total            <- ncol(metals)
comp_cols          <- setdiff(seq_len(K_total), benchmark_col)
forecast_variance  <- estimate_forecast_variance(metals,
                        benchmark_col = benchmark_col)
forecast_sd_models <- sqrt(forecast_variance[, comp_cols])
threshold_val      <- quantile(metals[, benchmark_col], 0.10)
zp_loss <- compute_zp(metals, forecast_sd_models,
                      threshold     = threshold_val,
                      benchmark_col = benchmark_col)
head(zp_loss)

RCtest documentation built on June 2, 2026, 9:07 a.m.