plot_performance_metrics: Plot Performance Metrics Comparison

In RCtest: Reality Check and Predictive Ability Tests for Forecast Evaluation

Plot Performance Metrics Comparison

Description

Generates a grid of scatter plots for Root Mean Squared Error (RMSE), Normalized Root Mean Squared Error (N-RMSE), Mean Absolute Error (MAE), and Mean Absolute Scaled Error (MASE) plotted against the Equally Weighted Risk Contribution (ERC) Weight of each competing forecast. These metrics are used for visualization only and are computed directly from forecast errors; they are distinct from the loss functions used in the WRC/SPA/CPA hypothesis tests (see run_comprehensive_erc_analysis).

Each panel shows:

• Points: one per model. The radius (size) of each circle is proportional to the model's Risk Contribution, defined as \text{RMSE}_k \times \text{ERC Weight}_k. Larger circles indicate forecasts that contribute more to the portfolio-level risk.

• Dashed line: an OLS regression line of the error metric (x-axis) on the ERC Weight (y-axis), showing whether higher-weighted forecasts tend to have systematically lower or higher error.

Note on MASE scaling: The MASE denominator used here is mean(abs(diff(benchmark))), where benchmark is the benchmark forecast column extracted from forecast_matrix. This differs from the scaling used in run_comprehensive_erc_analysis, where the denominator is mean(abs(diff(realizations_raw))) computed from the actual realised values. The two denominators coincide when the benchmark is a random-walk or historical-average forecast whose one-step-ahead forecasts equal the lagged realised value, but will diverge otherwise. The plot_performance_metrics function does not accept a separate realisation vector, so the benchmark forecast series is used as a proxy for the naive forecast scale. MASE values produced by this function are therefore intended for visual comparison across forecasts only and should not be directly compared to MASE values from the hypothesis tests in run_comprehensive_erc_analysis.

Usage

plot_performance_metrics(
  forecast_matrix,
  weights = NULL,
  benchmark_col = ncol(forecast_matrix)
)

Arguments

forecast_matrix	Matrix or data frame of dimension P x K_total, where P is the number of forecast periods, columns 1 to K_total - 1 are competing model forecasts, and the last column (or benchmark_col) is the benchmark.
weights	Optional numeric vector of length K_total - 1 giving ERC weights for each competing forecast. If NULL (default), equal weights 1/K are used.
benchmark_col	Index or name of the benchmark column. Defaults to the last column.

Value

A gtable object produced by grid.arrange containing four panels arranged in a 2x2 grid: RMSE (top-left), N-RMSE (top-right), MAE (bottom-left), MASE (bottom-right). Each panel is a ggplot object and can be extracted individually if needed.

Equally Weighted Risk Contribution (ERC) Weight

ERC weights are portfolio weights assigned so that every competing forecast contributes an equal share to the total portfolio risk (measured here by forecast error dispersion). Formally, weights w_k are chosen so that w_k \cdot \sigma_k = c for all k, where \sigma_k is a measure of forecast k's risk and c = \frac{1}{K}\sum_{k=1}^{K} w_k \sigma_k is the common per-forecast risk budget determined endogenously by the equal-contribution constraint (Maillard et al., 2010). When weights are supplied by the user, they are treated as pre-computed ERC weights and normalised to sum to one. When weights = NULL, equal weights 1/K are used as a baseline.

References

Maillard, S., Roncalli, T., & Teïletche, J. (2010). The Properties of Equally Weighted Risk Contributions Portfolios. The Journal of Portfolio Management, 36(4), 60–70. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3905/jpm.2010.36.4.060")}

See Also

run_comprehensive_erc_analysis, plot_cumulative_loss

Examples

data(metals)
K_models       <- ncol(metals) - 1
custom_weights <- (K_models:1) / sum(K_models:1)
plot_performance_metrics(metals, weights = custom_weights, benchmark_col = 15)

RCtest documentation built on June 2, 2026, 9:07 a.m.