modeltime_resample_accuracy: Calculate Accuracy Metrics from Modeltime Resamples

In modeltime.resample: Resampling Tools for Time Series Forecasting

Calculate Accuracy Metrics from Modeltime Resamples

Description

This is a wrapper for yardstick that simplifies time series regression accuracy metric calculations from a Modeltime Table that has been resampled and fitted using modeltime_fit_resamples().

Usage

modeltime_resample_accuracy(
  object,
  summary_fns = mean,
  metric_set = default_forecast_accuracy_metric_set(),
  ...
)

Arguments

object	a Modeltime Table with a column '.resample_results' (the output of modeltime_fit_resamples())
summary_fns	One or more functions to analyze resamples. The default is mean(). Possible values are: NULL, to returns the resamples untransformed. A function, e.g. mean. A purrr-style lambda, e.g. ~ mean(.x, na.rm = TRUE) A list of functions/lambdas, e.g. list(mean = mean, sd = sd)
metric_set	A yardstick::metric_set() that is used to summarize one or more forecast accuracy (regression) metrics.
...	Additional arguments passed to the function calls in summary_fns.

Details

#' Default Accuracy Metrics

The following accuracy metrics are included by default via modeltime::default_forecast_accuracy_metric_set():

• MAE - Mean absolute error, yardstick::mae()

• MAPE - Mean absolute percentage error, yardstick::mape()

• MASE - Mean absolute scaled error, yardstick::mase()

• SMAPE - Symmetric mean absolute percentage error, yardstick::smape()

• RMSE - Root mean squared error, yardstick::rmse()

• RSQ - R-squared, yardstick::rsq()

Summary Functions

By default, modeltime_resample_accuracy() returns the average accuracy metrics for each resample prediction.

The user can change this default behavior using summary_fns. Simply pass one or more Summary Functions. Internally, the functions are passed to dplyr::across(.fns), which applies the summary functions.

Returning Unsummarized Results

You can pass summary_fns = NULL to return unsummarized results by .resample_id.

Professional Tables (Interactive & Static)

Use modeltime::table_modeltime_accuracy() to format the results for reporting in reactable (interactive) or gt (static) formats, which are perfect for Shiny Apps (interactive) and PDF Reports (static).

Examples

library(modeltime)

# Mean (Default)
m750_training_resamples_fitted %>%
    modeltime_resample_accuracy() %>%
    table_modeltime_accuracy(.interactive = FALSE)

# Mean and Standard Deviation
m750_training_resamples_fitted %>%
    modeltime_resample_accuracy(
        summary_fns = list(mean = mean, sd = sd)
    ) %>%
    table_modeltime_accuracy(.interactive = FALSE)

# When summary_fns = NULL, returns the unsummarized resample results
m750_training_resamples_fitted %>%
    modeltime_resample_accuracy(
        summary_fns = NULL
    )

modeltime.resample documentation built on April 14, 2023, 12:31 a.m.