EvalTrainIndices: Generate Training and Evaluation Indices for Cross-Validation

In CSTools: Assessing Skill of Climate Forecasts on Seasonal-to-Decadal Timescales

Generate Training and Evaluation Indices for Cross-Validation

Description

This function generates training and evaluation indices based on different cross-validation methods.

Usage

EvalTrainIndices(
  eval.method,
  sample.length,
  sample.length_cor,
  k = 1,
  tail.out = TRUE
)

Arguments

eval.method	Character. The cross-validation method. Options include: -leave-k-out: Leaves out k points at a time for evaluation. -retrospective: Uses past data for training and a future point for evaluation. -in-sample: Uses the entire dataset for both training and evaluation. -hindcast-vs-forecast: Uses all years from the hindcast sdate dimension as training and all years from the forecast sdate dimension will be corrected.
sample.length	Integer. Length of the sample (in years).
sample.length_cor	Integer. Length of forecast sample (in years) in hindcast-vs-forecast method.
k	Positive integer. Default = 1. In method leave-k-out, k is expected to be odd integer, indicating the number of points to leave out. In method retrospective, k can be any positive integer, indicating when to start.
tail.out	Logical for method leave-k-out. Default = TRUE. TRUE to remove both extremes keeping the same sample size for all k-folds (e.g. sample.length=50, k=3, eval.dexes=1, train.dexes=(3,49)). FALSE to remove only the corresponding tail (e.g. sample.length=50, k=3, eval.dexes=1, train.dexes=(3,50))

Value

A list of lists, where each element contains: eval.dexes: Indices of evaluation points. train.dexes: Indices of training points.

Author(s)

Theertha Kariyathan, [e-mail]

Examples

# Leave-k-out cross-validation
EvalTrainIndices("leave-k-out", sample.length = 10, sample.length_cor = 5, k = 3)
# Retrospective cross-validation
EvalTrainIndices("retrospective", sample.length = 10, sample.length_cor = 5, k = 3)
# In-sample validation
EvalTrainIndices("in-sample", sample.length = 10, sample.length_cor = 5)
# Hindcast vs. Forecast validation
EvalTrainIndices("hindcast-vs-forecast", sample.length = 10, sample.length_cor = 5)

CSTools documentation built on Nov. 14, 2025, 5:07 p.m.