In ajmcneil/spectralBacktest: Spectral Backtests of Forecast Distributions

library(spectralBacktest)
library(purrr)
library(knitr)
set.seed(543)

The spectral MD-test is called with the function spectral_MDtest(). It currently handles monospectral and bispectral tests. Extension to multispectral tests is in progress.

First we draw a sample of PIT values by applying the normal CDF to a Gaussian ARMA process. The PIT values should display little serial correlation, but serial dependence is revealed in the ACF of the transformed variable $|2P-1|$. The rHS_arma function emulates historical simulation when there is unmodelled ARMA(1,1) dependence in factor returns.

n <- 500

P <- rHS_arma(n, list(ar=0.35, ma=-0.1))
acf(P,lag=5)
acf(abs(2*P-1),lag=5)

Samples generated by rHS_arma should be close to uniform in unconditional distribution. We verify with a set of spectral Z-tests.

alpha1 <- 0.95
alpha_star <- 0.99
alpha2 <- 0.995


ZU <- list( name = 'Uniform',
            type = 'mono',
            nu = nu_uniform,
            support = c(alpha1, alpha2),
            param = NULL )

ZLL <- list( name = 'Linear/Linear',
             type = 'bi',
             nu = list(nu_linear, nu_linear),
             correlation = rho_linear_linear,
             support = c(alpha1, alpha2),
             param = list(-1, 1) )

PNS <- list( name = 'Probitnormal score',
             type = 'bi',
             nu = list(nu_probitnormal, nu_probitnormal),
             correlation = rho_probitnormal,
             support = c(alpha1, alpha2),
             param = list(1L, 2L) )

# gather the tests into a list and execute!
kernlist <- list(ZU=ZU, ZLL=ZLL, PNS=PNS)
pval_Z <- map_df(kernlist, function(kern) spectral_Ztest(kern,P))
kable(pval_Z, digits=5, caption='p-values of tests of unconditional coverage')

The MD tests require that we specify a kernel and a CVT structure with the same dimensionality. That is, we use a monospectral CVT with a monokernel and a bispectral CVT with a bikernel. We want to run a set of tests, so specify a lists of these structures. The MD tests offer three ways to calculate the $H$ matrix, so each is demonstrated.

# Monospectral
CVT <- list( name = 'Power 4',
             type = 'mono',
             h_closure = CVT_PtildePower,
             h_param = 4,
             lags = 4L )

# Bispectral.  Note that each element of lags is a nonnegative integer.
CVT2 <- list( name = 'Power4/Power 0.5',
              type = 'bi',
              h_closure = list(CVT_PtildePower, CVT_PtildePower),
              correlation = rho_PtildePower,
              h_param = list(4,1/2),  
              lags = c(4L,0) )

MDlist <- list(CVT, CVT2, CVT2)  # same length as kernlist, defined earlier

infoinH <- list('none', 'partial', 'full')
pval_MD <- map_df(infoinH, function(s) 
                       q <- c(Information=s,
                              map2(kernlist, MDlist, 
                              ~spectral_MDtest(.x, .y, P, informationH=s))))

kable(pval_MD, digits=5, caption='p-values of MD tests of conditional coverage')