tsbacktest: Walk Forward Model Backtest

tsbacktest.tsissm.specR Documentation

Walk Forward Model Backtest

Description

Generates an expanding window walk forward backtest.

Usage

## S3 method for class 'tsissm.spec'
tsbacktest(
  object,
  start = floor(length(object$target$y_orig)/2),
  end = length(object$target$y_orig),
  h = 1,
  estimate_every = 1,
  FUN = NULL,
  alpha = NULL,
  solver = "nlminb",
  autodiff = TRUE,
  autoclean = FALSE,
  trace = FALSE,
  ...
)

Arguments

object

an object of class “tsissm.spec”.

start

numeric data index from which to start the backtest.

end

numeric data index on which to end the backtest. The backtest will end 1 period before that date in order to have at least 1 out of sample value to compare against.

h

forecast horizon. As the expanding window approaches the “end”, the horizon will automatically shrink to the number of available out of sample periods.

estimate_every

number of periods at which the model is re-estimated and new predictions are generated (defaults to 1).

FUN

optional function which is applied across all horizons for each draw (i.e. operating on each row of the distribution which represents a single path). For example, using the max function will return the distribution of the maximum across all horizons, whereas sum (for flow variables) would represent the aggregate value distribution. The P50 of this distribution is returned and aligned with the terminal horizon for each re-estimation period, and if alpha is not NULL, then the quantiles of this distributions with respect to the coverage (alpha) chosen.

alpha

optional numeric vector of coverage rates for which to calculate the quantiles.

solver

solver to use.

autodiff

whether to use automatic differentiation for estimation. This makes use of the tsissmad package.

autoclean

whether to perform automatic cleaning on the training data prior to prediction as per the ‘auto_clean’ function in the tsaux package.

trace

whether to show the progress bar. The user is expected to have set up appropriate handlers for this using the “progressr” package.

...

additional arguments passed to the “auto_clean” function.

Value

A list with the following data.tables:

  • prediction : the backtest table with forecasts and actuals

  • metrics: a summary performance table showing metrics by forecast horizon (MAPE, MSLRE, BIAS and MIS if alpha was not NULL).

Note

The function can use parallel functionality as long as the user has set up a plan using the future package.


tsmodels/tsissm documentation built on June 22, 2022, 6:57 a.m.