pt_collect_results: Collect diagnostics from two 'ml_backtest_multi()' runs

In PortfolioTesteR: Test Investment Strategies with English-Like Code

Collect diagnostics from two ml_backtest_multi() runs

Description

Builds a compact set of outputs—coverage, IC series, OOS-only rolling IC, performance tables (Full/Pre/Post), turnover, and a cost sweep—given two lists of backtests (pooled and per-group) produced by ml_backtest_multi().

Usage

pt_collect_results(
  bt_pooled_list,
  bt_neutral_list,
  weekly_prices,
  horizons,
  split_date,
  cost_bps = c(5, 10, 15, 20, 25, 50),
  freq = 52,
  prices = NULL,
  ic_roll_window = 26L,
  mask_scores_to_decision_dates = TRUE,
  cost_model = function(turnover, bps) (bps/10000) * turnover
)

Arguments

bt_pooled_list	Named list of backtests (keys like H4w, H13w) produced with group_mode = "pooled".
bt_neutral_list	Named list of backtests (same keys) produced with group_mode = "per_group".
weekly_prices	Deprecated alias for prices; kept for backwards compatibility.
horizons	Integer vector of horizons (in weeks) expected in the lists.
split_date	Date used to split performance into Pre/Post.
cost_bps	Numeric vector of per-rebalance cost levels (in basis points) for the turnover-based cost sweep. Default c(5, 10, 15, 20, 25, 50).
freq	Integer frequency used by perf_metrics() (e.g., 52 for weekly).
prices	Optional price table (preferred). If NULL, weekly_prices is used.
ic_roll_window	Integer window length (weeks) for rolling IC on OOS decision dates. Default 26L.
mask_scores_to_decision_dates	Logical; if TRUE (default) scores are masked to OOS decision dates only (see scores_oos_only()).
cost_model	Function ⁠(turnover, bps)⁠ returning per-period cost to subtract from returns in the sweep. Default scales linearly with turnover.

Details

Both input lists must have identical horizon keys (paste0("H", h, "w")). Coverage and IC series are computed from stored scores; rolling IC is built on OOS decision dates only; performance is summarised for the full sample and Pre/Post relative to split_date; turnover is derived from realised sector-neutral weights; and a turnover-based cost sweep is evaluated on the sector-neutral run across cost_bps.

Value

A named list with one element per horizon, each containing:

• bt_pooled, bt_neutral — the input backtests;

• coverage — coverage by date for pooled/neutral;

• ic_series — raw IC series for pooled/neutral;

• icroll_oos_26w — rolling IC (OOS-only) for pooled/neutral;

• masked_scores — OOS-masked score tables for pooled/neutral;

• perf_tables — performance tables (Full/Pre/Post);

• turnover_neutral — turnover series for the sector-neutral run;

• cost_sweep_neutral — performance under gross/net across cost_bps.

See Also

ml_backtest_multi(), scores_oos_only(), perf_metrics()

Other Chapter3-helpers: scores_oos_only()

Examples

if (requireNamespace("PortfolioTesteR", quietly = TRUE)) {
  library(PortfolioTesteR)
  data(sample_prices_weekly, package = "PortfolioTesteR")
  # Simple feature
  mom12 <- PortfolioTesteR::calc_momentum(sample_prices_weekly, 12)
  feats <- list(mom12 = mom12)
  fit_first     <- function(X, y, ...) list()
  predict_first <- function(model, Xnew, ...) as.numeric(Xnew[[1]])
  sch <- list(is = 52L, oos = 26L, step = 26L)
  syms <- setdiff(names(sample_prices_weekly), "Date")
  gmap <- data.frame(Symbol = syms,
                     Group  = rep(c("G1","G2"), length.out = length(syms)))
  bt_pooled  <- ml_backtest_multi(feats, sample_prices_weekly, c(4L),
                                  fit_first, predict_first, sch,
                                  selection = list(top_k = 5L),
                                  weighting = list(method = "softmax", temperature = 12),
                                  caps = list(max_per_symbol = 0.10),
                                  group_mode = "pooled")
  bt_neutral <- ml_backtest_multi(feats, sample_prices_weekly, c(4L),
                                  fit_first, predict_first, sch,
                                  selection = list(top_k = 5L),
                                  weighting = list(method = "softmax", temperature = 12),
                                  caps = list(max_per_symbol = 0.10),
                                  group_mode = "per_group",
                                  group_map = gmap)
  out <- pt_collect_results(
    bt_pooled_list  = bt_pooled,
    bt_neutral_list = bt_neutral,
    prices          = sample_prices_weekly,
    horizons        = c(4L),
    split_date      = as.Date("2019-01-01"),
    cost_bps        = c(5, 15),
    freq            = 52,
    ic_roll_window  = 13L
  )
  names(out)
  str(out[["H4w"]]$perf_tables)
}

PortfolioTesteR documentation built on Nov. 5, 2025, 5:23 p.m.