MASE: Mean Absolute Scaled Error (MASE)

In metrica: Prediction Performance Metrics

Mean Absolute Scaled Error (MASE)

Description

It estimates the mean absolute error using the naive-error approach for a continuous predicted-observed dataset.

Usage

MASE(
  data = NULL,
  obs,
  pred,
  time = NULL,
  naive_step = 1,
  oob_mae = NULL,
  tidy = FALSE,
  na.rm = TRUE
)

Arguments

data	(Required) argument to call an existing data frame containing the data.
obs	Vector with observed values (numeric).
pred	Vector with predicted values (numeric).
time	(Optional) String with the "name" of the vector containing the time variable to sort observations. "Optional" to ensure an appropriate MASE estimation. Default: NULL, assumes observations are already sorted by time.
naive_step	A positive number specifying how many observed values to recall back in time when computing the naive expectation. Default = 1
oob_mae	A numeric value indicating the out-of-bag (out-of-sample) MAE. By default, an in-sample MAE is calculated using the naive forecast method. See Hyndman & Koehler (2006). Default : NULL.
tidy	Logical operator (TRUE/FALSE) to decide the type of return. TRUE returns a data.frame, FALSE returns a list; Default : FALSE.
na.rm	Logic argument to remove rows with missing values (NA). Default is na.rm = TRUE.

Details

The MASE is especially well suited for time series predictions. It can be used to compare forecast methods on a single series and also to compare forecast accuracy between series.

This metric never gives an infinite or undefined values (unless all observations over time are exactly the same!).

By default, the MASE scales the error based on in-sample MAE from the naive forecast method (random walk). The in-sample MAE is used in the denominator because it is always available and it effectively scales the errors.Since it is based on absolute error, it is less sensitive to outliers compared to the classic MSE.

MASE = \frac{1}{n}(\frac{|O_i-P_i|}{ \frac{1}{T-1} \sum^T_{t=2}~|O_t - O_{t-1}| })

If available, users may use and out-of-bag error from an independent dataset, which can be specified with the oob_mae arg. and will replace the denominator into the MASE equation.

MASE measures total error (i.e. both lack of accuracy and precision.). The lower the MASE below 1, the better the prediction quality. MASE = indicates no difference between the model and the naive forecast (or oob predictions). MASE > 1 indicate poor performance.

For the formula and more details, see online-documentation

Value

an object of class numeric within a list (if tidy = FALSE) or within a ⁠data frame⁠ (if tidy = TRUE).

References

Hyndman, R.J., Koehler, A.B. (2006). Another look at measures of forecast accuracy. _ Int. J. Forecast_ \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3354/cr030079")}

Examples

set.seed(1)
# Create a fake dataset
X <- rnorm(n = 100, mean = 8, sd = 10)
Y <- rnorm(n = 100, mean = 8.2, sd = 15)
Z <- seq(1, 100, by = 1)

time_data <- as.data.frame(list("observed" = X, "predicted" = Y, "time" = Z))
 
MASE(data = time_data, obs = observed, pred = predicted, time = time)

metrica documentation built on June 30, 2024, 5:07 p.m.