anomalize: Automatic group-wise Anomaly Detection

In business-science/timekit: A Tool Kit for Working with Time Series

Automatic group-wise Anomaly Detection

Description

anomalize() is used to detect anomalies in time series data, either for a single time series or for multiple time series grouped by a specific column.

Usage

anomalize(
  .data,
  .date_var,
  .value,
  .frequency = "auto",
  .trend = "auto",
  .method = "stl",
  .iqr_alpha = 0.05,
  .clean_alpha = 0.75,
  .max_anomalies = 0.2,
  .message = TRUE
)

Arguments

.data	A tibble or data.frame with a time-based column
.date_var	A column containing either date or date-time values
.value	A column containing numeric values
.frequency	Controls the seasonal adjustment (removal of seasonality). Input can be either "auto", a time-based definition (e.g. "2 weeks"), or a numeric number of observations per frequency (e.g. 10). Refer to tk_get_frequency().
.trend	Controls the trend component. For STL, trend controls the sensitivity of the LOESS smoother, which is used to remove the remainder. Refer to tk_get_trend().
.method	The outlier detection method. Default: "stl". Currently "stl" is the only method. "twitter" is planned.
.iqr_alpha	Controls the width of the "normal" range. Lower values are more conservative while higher values are less prone to incorrectly classifying "normal" observations.
.clean_alpha	Controls the threshold for cleaning the outliers. The default is 0.75, which means that the anomalies will be cleaned using the 0.75 * lower or upper bound of the recomposed time series, depending on the direction of the anomaly.
.max_anomalies	The maximum percent of anomalies permitted to be identified.
.message	A boolean. If TRUE, will output information related to automatic frequency and trend selection (if applicable).

Details

The anomalize() method for anomaly detection that implements a 2-step process to detect outliers in time series.

Step 1: Detrend & Remove Seasonality using STL Decomposition

The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder" for anomaly detection.

The user can control two parameters: frequency and trend.

1. .frequency: Adjusts the "season" component that is removed from the "observed" values.

2. .trend: Adjusts the trend window (t.window parameter from stats::stl() that is used.

The user may supply both .frequency and .trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or trend based on the scale of the time series using the tk_time_scale_template().

Step 2: Anomaly Detection

Once "trend" and "season" (seasonality) is removed, anomaly detection is performed on the "remainder". Anomalies are identified, and boundaries (recomposed_l1 and recomposed_l2) are determined.

The Anomaly Detection Method uses an inner quartile range (IQR) of +/-25 the median.

IQR Adjustment, alpha parameter

With the default alpha = 0.05, the limits are established by expanding the 25/75 baseline by an IQR Factor of 3 (3X). The IQR Factor = 0.15 / alpha (hence 3X with alpha = 0.05):

• To increase the IQR Factor controlling the limits, decrease the alpha, which makes it more difficult to be an outlier.

• Increase alpha to make it easier to be an outlier.

• The IQR outlier detection method is used in forecast::tsoutliers().

• A similar outlier detection method is used by Twitter's AnomalyDetection package.

• Both Twitter and Forecast tsoutliers methods have been implemented in Business Science's anomalize package.

Value

A tibble or data.frame with the following columns:

• observed: original data

• seasonal: seasonal component

• seasadaj: seasonal adjusted

• trend: trend component

• remainder: residual component

• anomaly: Yes/No flag for outlier detection

• anomaly score: distance from centerline

• anomaly direction: -1, 0, 1 inidicator for direction of the anomaly

• recomposed_l1: lower level bound of recomposed time series

• recomposed_l2: upper level bound of recomposed time series

• observed_clean: original data with anomalies interpolated

References

1. CLEVELAND, R. B., CLEVELAND, W. S., MCRAE, J. E., AND TERPENNING, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, Vol. 6, No. 1 (1990), pp. 3-73.

2. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014). A Novel Technique for Long-Term Anomaly Detection in the Cloud. Twitter Inc.

Examples

library(dplyr)

walmart_sales_weekly %>%
    filter(id %in% c("1_1", "1_3")) %>%
    group_by(id) %>%
    anomalize(Date, Weekly_Sales)

business-science/timekit documentation built on Feb. 2, 2024, 2:51 a.m.