make_accuracy: Estimate point forecast accuracy

In tscv: Functions and Utilities for Tidy Time Series Forecasting and Time Series Cross-Validation

Estimate point forecast accuracy

Description

Estimate accuracy metrics for point forecasts generated from rolling-origin time series cross-validation.

Usage

make_accuracy(
  future_frame,
  main_frame,
  context,
  dimension = "split",
  benchmark = NULL
)

Arguments

future_frame	A tibble containing point forecasts. It must contain the columns specified by context, as well as model, split, horizon, and point.
main_frame	A tibble containing the observed values. It must contain the series identifier, time index, and value column specified by context.
context	A named list with the identifiers for series_id, value_id, and index_id.
dimension	Character value. Determines the dimension over which accuracy is summarized. Common choices are "split" and "horizon".
benchmark	Optional character value giving the model name used as the benchmark for the relative mean absolute error rMAE.

Details

make_accuracy() compares point forecasts in future_frame with the observed values in main_frame. The two data sets are joined using the series identifier and time index defined in context.

Accuracy can be summarized along different cross-validation dimensions:

• dimension = "split" summarizes accuracy separately for each test split.

• dimension = "horizon" summarizes accuracy separately for each forecast horizon.

The following point forecast accuracy metrics are returned:

• ME: mean error.

• MAE: mean absolute error.

• MSE: mean squared error.

• RMSE: root mean squared error.

• MAPE: mean absolute percentage error.

• sMAPE: symmetric mean absolute percentage error.

• MPE: mean percentage error.

If benchmark is supplied, the function also computes the relative mean absolute error rMAE. The rMAE is calculated as the model's MAE divided by the MAE of the benchmark model for the same series and selected dimension.

Value

A tibble containing the forecast accuracy metrics. The output contains the series identifier, model name, selected dimension, dimension value n, metric name, and metric value.

See Also

Other accuracy functions: mae_vec(), make_errors(), mape_vec(), me_vec(), mpe_vec(), mse_vec(), rmse_vec(), smape_vec()

Examples

library(dplyr)
library(tsibble)
library(fabletools)
library(fable)

context <- list(
  series_id = "series",
  value_id = "value",
  index_id = "index"
)

main_frame <- M4_monthly_data |>
  filter(series %in% c("M23100", "M14395"))

split_frame <- make_split(
  main_frame = main_frame,
  context = context,
  type = "first",
  value = 120,
  n_ahead = 18,
  n_skip = 17,
  n_lag = 0,
  mode = "stretch",
  exceed = FALSE
)

train_frame <- slice_train(
  main_frame = main_frame,
  split_frame = split_frame,
  context = context
) |>
  as_tsibble(
    index = index,
    key = c(series, split)
  )

model_frame <- train_frame |>
  model(
    "SNAIVE" = SNAIVE(value ~ lag("year"))
  )

fable_frame <- model_frame |>
  forecast(h = 18)

future_frame <- make_future(
  fable = fable_frame,
  context = context
)

accuracy_horizon <- make_accuracy(
  future_frame = future_frame,
  main_frame = main_frame,
  context = context,
  dimension = "horizon"
)

accuracy_horizon

accuracy_split <- make_accuracy(
  future_frame = future_frame,
  main_frame = main_frame,
  context = context,
  dimension = "split"
)

accuracy_split

tscv documentation built on May 13, 2026, 9:07 a.m.