tsbacktest: Walk Forward Model Backtest

In tsmodels/tsissm: Linear Innovations State Space Unobserved Components Model

Walk Forward Model Backtest

Description

Generates an expanding window walk forward backtest.

Usage

## S3 method for class 'tsissm.spec'
tsbacktest(
  object,
  start = floor(length(object$target$y_orig)/2),
  end = length(object$target$y_orig),
  h = 1,
  estimate_every = 1,
  FUN = NULL,
  alpha = NULL,
  solver = "nlminb",
  autodiff = TRUE,
  autoclean = FALSE,
  trace = FALSE,
  ...
)

Arguments

object	an object of class “tsissm.spec”.
start	numeric data index from which to start the backtest.
end	numeric data index on which to end the backtest. The backtest will end 1 period before that date in order to have at least 1 out of sample value to compare against.
h	forecast horizon. As the expanding window approaches the “end”, the horizon will automatically shrink to the number of available out of sample periods.
estimate_every	number of periods at which the model is re-estimated and new predictions are generated (defaults to 1).
FUN	optional function which is applied across all horizons for each draw (i.e. operating on each row of the distribution which represents a single path). For example, using the max function will return the distribution of the maximum across all horizons, whereas sum (for flow variables) would represent the aggregate value distribution. The P50 of this distribution is returned and aligned with the terminal horizon for each re-estimation period, and if alpha is not NULL, then the quantiles of this distributions with respect to the coverage (alpha) chosen.
alpha	optional numeric vector of coverage rates for which to calculate the quantiles.
solver	solver to use.
autodiff	whether to use automatic differentiation for estimation. This makes use of the tsissmad package.
autoclean	whether to perform automatic cleaning on the training data prior to prediction as per the ‘auto_clean’ function in the tsaux package.
trace	whether to show the progress bar. The user is expected to have set up appropriate handlers for this using the “progressr” package.
...	additional arguments passed to the “auto_clean” function.

Value

A list with the following data.tables:

• prediction : the backtest table with forecasts and actuals

• metrics: a summary performance table showing metrics by forecast horizon (MAPE, MSLRE, BIAS and MIS if alpha was not NULL).

Note

The function can use parallel functionality as long as the user has set up a plan using the future package.

tsmodels/tsissm documentation built on Oct. 15, 2022, 6:44 a.m.