superior_predictive_ability_test: Superior Predictive Ability (SPA) Test

In RCtest: Reality Check and Predictive Ability Tests for Forecast Evaluation

Superior Predictive Ability (SPA) Test

Description

Implements the Hansen (2005) Superior Predictive Ability (SPA) test, a studentized extension of White's (2000) Reality Check that corrects for the inclusion of irrelevant (poor) forecasts to reduce conservatism.

Hypotheses:

• H0: \max_{k} E[g(u_{0,t}) - g(u_{k,t})] \leq 0 – no competing forecast produces strictly lower expected loss than the benchmark forecast.

• H1: At least one competing forecast has strictly lower expected loss than the benchmark forecast.

Usage

superior_predictive_ability_test(
  loss_differences,
  block_length,
  num_bootstrap_replications,
  alpha
)

Arguments

loss_differences	A numeric matrix (P x K) of loss differences (benchmark loss minus forecast loss).
block_length	integer. The block length for MBB and HAC estimation. A commonly used rule of thumb is T^{1/3} (Politis & Romano, 1994). For P = 165, this gives approximately 5–6.
num_bootstrap_replications	integer number of MBB bootstrap replications. Default 999; use at least 999 for reliable inference (Davidson & MacKinnon, 2000).
alpha	numeric. The significance level (default 0.05).

Details

The SPA statistic studentizes each mean loss differential by its HAC standard deviation (estimated via estimate_long_run_covariance), then takes the maximum across forecasts. Two p-values are returned, corresponding to two choices of the null distribution (Hansen, 2005, Section 3):

• p_consistent: uses the sample-dependent null estimator \hat{\mu}^c, which recentres the bootstrap statistic at the sample mean \bar{d}_k for each forecast. This is the recommended p-value.

• p_conservative: uses the Least Favourable Configuration (LFC) \hat{\mu}^u = 0 for all forecasts – equivalent to White's (2000) Reality Check bootstrap, where no recentring is applied. This provides an upper bound on the true p-value and is always \geq p_consistent.

Value

An object of class "htest". Additionally contains p_consistent and p_conservative for the two SPA bootstrap variants.

References

Hansen, P. R. (2005). A Test for Superior Predictive Ability. Journal of Business & Economic Statistics, 23(4), 365–380. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/073500105000000063")}

Corradi, V., & Swanson, N. R. (2011). The White Reality Check and some of its recent extensions. In Festschrift in honor of Halbert L. White.

Davidson, R., & MacKinnon, J. G. (2000). Bootstrap tests: How many bootstraps? Econometric Reviews, 19(1), 55–68. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/07474930008800459")}

Kunsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3), 1217–1241. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/aos/1176347265")}

Examples

data(metals)
# metals: 165 x 15; columns 1-14 are competing forecasts, column 15 is the benchmark
# A small offset (+0.5) is added to the lagged benchmark to avoid degenerate zero
# loss differences when forecasts equal the realized value exactly (illustration only).
P <- nrow(metals)
K_total <- ncol(metals)
K <- K_total - 1 # 14 competing forecasts
realized       <- c(metals[-1, K_total], metals[P, K_total]) + 0.5
benchmark_loss <- (metals[, K_total] - realized)^2
model_loss     <- (metals[, 1:K] - realized)^2
loss_diff      <- benchmark_loss - model_loss
res <- superior_predictive_ability_test(loss_diff, block_length = 5,
                                        num_bootstrap_replications = 50,
                                        alpha = 0.05)
print(res)

RCtest documentation built on June 2, 2026, 9:07 a.m.