exdqlmForecastDiagnostics: Held-out forecast diagnostics for exDQLM forecasts

In exdqlm: Extended Dynamic Quantile Linear Models

Held-out forecast diagnostics for exDQLM forecasts

Description

Computes held-out forecast scores from one or two exdqlmForecast objects returned by exdqlmForecast(). Unlike exdqlmDiagnostics(), which summarizes fitted one-step-ahead forecast errors and their KL normality diagnostics, this function evaluates posterior predictive forecast draws against observations reserved outside the fitted sample.

Usage

exdqlmForecastDiagnostics(
  m1,
  m2 = NULL,
  y,
  p0 = NULL,
  crps_probs = seq(0.01, 0.99, by = 0.01),
  crps_weights = NULL
)

Arguments

m1	An object of class "exdqlmForecast", returned by exdqlmForecast() with return.draws = TRUE.
m2	An optional second object of class "exdqlmForecast" to compare with m1.
y	Numeric vector or time series of held-out observations. Its length must equal the forecast horizon.
p0	Optional quantile level used for the check-loss calculation. When NULL, the value is taken from m1$m1$p0. If m2 is supplied, its fitted quantile level must agree with the resolved value.
crps_probs	Numeric vector of quantile levels used to approximate CRPS through the integrated quantile-score identity. Values must be strictly between 0 and 1. Default is seq(0.01, 0.99, by = 0.01).
crps_weights	Optional non-negative numeric weights for crps_probs. When NULL, equal weights are used. When provided, weights are normalized to sum to 1.

Details

The check loss is computed at the target quantile level p0 using the forecast quantile means ff stored in each forecast object. CRPS is computed from samp.fore using the same finite integrated quantile-score approximation used by exdqlmDiagnostics(). This function does not compute KL diagnostics because KL in exdqlm is defined for fitted one-step-ahead MAP standardized forecast errors, not for arbitrary held-out forecast draws.

Value

An object of class "exdqlmForecastDiagnostic" containing:

• y - Held-out observations used for scoring.

• p0 - Quantile level used for check loss.

• horizon - Forecast horizon.

• m1.check_loss - Mean target-quantile check loss for m1.

• m1.CRPS - Mean CRPS approximation for m1.

• m1.pointwise - Pointwise held-out scores for m1.

• crps.method, crps.probs, and crps.weights - CRPS approximation metadata.

If m2 is supplied, analogous m2.* fields are included.

Examples

data("scIVTmag", package = "exdqlm")
old = options(exdqlm.max_iter = 15L)
y = scIVTmag[1:65]
y_train = y[1:60]
y_holdout = y[61:65]
model = polytrendMod(1, stats::quantile(y_train, 0.85), 10)
M0 = exdqlmLDVB(y_train, p0 = 0.85, model, df = c(0.98), dim.df = c(1),
                 gam.init = -3.5, sig.init = 15,
                 n.samp = 20, tol = 0.2, verbose = FALSE)
fFF = model$FF[, 1, drop = FALSE]
fGG = model$GG
M0.forecast = exdqlmForecast(start.t = 60, k = 5, m1 = M0,
                             fFF = fFF, fGG = fGG,
                             return.draws = TRUE, n.samp = 20, seed = 123,
                             plot = FALSE)
exdqlmForecastDiagnostics(M0.forecast, y = y_holdout)
options(old)

exdqlm documentation built on June 5, 2026, 1:06 a.m.