white_reality_check_conditional: Conditional Predictive Ability (CPA) Reality Check Test

In RCtest: Reality Check and Predictive Ability Tests for Forecast Evaluation

Conditional Predictive Ability (CPA) Reality Check Test

Description

Implements the Conditional Predictive Ability (CPA) test of Giacomini & White (2006), extended to a multiple-forecast setting via a studentized Reality Check statistic. Tests whether any competing forecast's predictive advantage over the benchmark is state-dependent, i.e., predictable from a conditioning variable h_t known at the time the forecast is made.

Hypotheses:

• H0: E[h_t \cdot (g(u_{0,t}) - g(u_{k,t}))] = 0 for all k = 1,\ldots,K – no competing forecast's loss differential with the benchmark is predictable using the conditioning information h_t.

• H1: At least one forecast's loss differential d_{k,t} = g(u_{0,t}) - g(u_{k,t}) is predictable by h_t, i.e., E[h_t \cdot d_{k,t}] \neq 0 for some k.

Usage

white_reality_check_conditional(
  loss_differences,
  weighting_vector,
  block_length,
  num_bootstrap_replications,
  alpha
)

Arguments

loss_differences	A numeric matrix (P x K) of loss differences (benchmark loss minus forecast loss), where P is the number of forecast periods and K is the number of competing forecasts. A positive entry means the competing forecast outperforms the benchmark forecast in that period.
weighting_vector	numeric vector of length P serving as the conditioning instrument h_t in the CPA test. At each period t, the test checks whether the loss differential d_{k,t} covaries with h_t, i.e., whether E[h_t \cdot d_{k,t}] \neq 0. See the Conditioning Instrument section in Details for interpretation, requirements, and recommended choices.
block_length	integer. The block length for MBB and HAC estimation. A commonly used rule of thumb is T^{1/3} (Politis & Romano, 1994). For P = 165, this gives approximately 5–6.
num_bootstrap_replications	integer number of MBB bootstrap replications. Default 999; see Davidson & MacKinnon (2000).
alpha	numeric. The significance level (default 0.05).

Details

The test multiplies each column of loss_differences element-wise by weighting_vector to form the weighted loss differential series h_t \cdot d_{k,t}. The unconditional mean of this product, E[h_t \cdot d_{k,t}], equals zero under H0 by the law of iterated expectations when h_t is a valid instrument. The test statistic is the maximum studentized mean across all K forecasts:

\hat{T}_{CPA} = \max_{k} \frac{\frac{1}{P}\sum_t h_t d_{k,t}} {\hat{\sigma}_{k,h}}

where \hat{\sigma}_{k,h} is the HAC standard deviation of h_t d_{k,t} estimated via estimate_long_run_covariance. Bootstrap p-values are obtained via the MBB of Kunsch (1989) with recentring, following the SPA-type procedure of Hansen (2005) applied to the weighted series.

Conditioning Instrument (weighting_vector)

A significant result means that knowing h_t allows one to predict which forecast will perform better in period t – the benchmark's advantage (or disadvantage) is state-dependent and potentially exploitable. This is a strictly stronger statement than the unconditional WRC: a forecast can fail the WRC (no unconditional improvement) yet pass the CPA test (conditional improvement in specific states).

h_t must be measurable with respect to the information set available at time t (Giacomini & White, 2006, Assumption 1) – it must not use information from period t+1 or later. The scale of h_t does not affect the test result because the statistic is studentized by its own HAC standard deviation.

Recommended choices:

abs(realized) – absolute realised values

Tests whether forecast performance depends on outcome magnitude – a natural proxy for market volatility or economic uncertainty. Default in run_comprehensive_erc_analysis.

c(realized[1], realized[-length(realized)]) – lagged realised values

Tests whether the previous period's outcome predicts which forecast wins next period. Relevant when forecast errors are autocorrelated.

rep(1, P) – constant vector

The product h_t \cdot d_{k,t} reduces to d_{k,t}, making the CPA test equivalent to the unconditional WRC. Use as a sanity check: results should be consistent with white_reality_check.

External economic indicator

E.g., a recession dummy, VIX level, lagged interest rate spread, or monetary policy stance dummy. Tests whether one forecast systematically outperforms during specific regimes. Must be lagged one period to ensure h_t is in the information set at the time of the forecast.

The scale of weighting_vector has no effect on inference because both the test statistic and its bootstrap distribution are studentized by the same \hat{\sigma}_{k,h}.

Value

An object of class "htest" with the following components:

statistic	Maximum studentized weighted mean loss differential across all K forecasts, labelled "T-CPA".
p.value	Bootstrap p-value from the MBB procedure.
method	"Conditional Predictive Ability (CPA) Test".
null.value	Named scalar "max studentized weighted mean loss differential" = 0.
alternative	Direction of the alternative hypothesis.
reject_null	Logical: TRUE if p.value <= alpha.

A small p-value indicates that at least one forecast's loss differential is predictable from the conditioning variable h_t. Failure to reject H0 means no evidence of state-dependent predictive ability for the chosen instrument.

References

Giacomini, R., & White, H. (2006). Tests of Conditional Predictive Ability. Econometrica, 74(6), 1545–1578. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1111/j.1468-0262.2006.00718.x")}

Hansen, P. R. (2005). A Test for Superior Predictive Ability. Journal of Business & Economic Statistics, 23(4), 365–380. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/073500105000000063")}

Corradi, V., & Swanson, N. R. (2011). The White Reality Check and some of its recent extensions. In Festschrift in honor of Halbert L. White.

Davidson, R., & MacKinnon, J. G. (2000). Bootstrap tests: How many bootstraps? Econometric Reviews, 19(1), 55–68. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/07474930008800459")}

Kunsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3), 1217–1241. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/aos/1176347265")}

Politis, D. N., & Romano, J. P. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428), 1303–1313. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/01621459.1994.10476870")}

See Also

white_reality_check for the unconditional WRC test (equivalent to CPA with a constant weighting_vector); superior_predictive_ability_test for the studentized unconditional test.

Examples

data(metals)
# metals: 165 x 15; columns 1-14 are competing forecasts, column 15 is the benchmark
# A small offset (+0.5) is added to the lagged benchmark to avoid degenerate zero
# loss differences when forecasts equal the realized value exactly (illustration only).
P       <- nrow(metals)
K_total <- ncol(metals)
K       <- K_total - 1L  # 14 competing forecasts
realized       <- c(metals[-1, K_total], metals[P, K_total]) + 0.5
benchmark_loss <- (metals[, K_total] - realized)^2
model_loss     <- (metals[, 1:K]     - realized)^2
loss_diff      <- benchmark_loss - model_loss

# Example 1: absolute realised values as conditioning variable (volatility proxy)
res1 <- white_reality_check_conditional(
  loss_differences           = loss_diff,
  weighting_vector           = abs(realized),
  block_length               = 5,
  num_bootstrap_replications = 50,
  alpha                      = 0.05
)
print(res1)

# Example 2: constant vector - should give results consistent with white_reality_check()
res2 <- white_reality_check_conditional(
  loss_differences           = loss_diff,
  weighting_vector           = rep(1, P),
  block_length               = 5,
  num_bootstrap_replications = 50,
  alpha                      = 0.05
)
print(res2)

RCtest documentation built on June 2, 2026, 9:07 a.m.