ReduceAnomalies: Reduce Anomalies

In alaineiturria/otsad: Online Time Series Anomaly Detectors

Reduce Anomalies

Description

ReduceAnomalies It reduces the number of detected anomalies. This function is designed to reduce the number of false positives keeping only the first detection of all those that are close to each other. This proximity distance is defined by a window

Usage

ReduceAnomalies(data, windowLength, incremental = FALSE, last.res = NULL)

Arguments

data	Numerical vector with anomaly labels.
windowLength	Window length.
incremental	TRUE for incremental processing and FALSE for classic processing
last.res	Last result returned by the algorithm.

Value

If incremental = FALSE, new Numerical vector with reduced anomaly labels. Else, a list of the following items.

result	New Numerical vector with reduced anomaly labels.
last.res	Last result returned by the algorithm. It is a list with pointer, the index of the last anomaly and index, the index number of the last point in the data

Examples

## EXAMPLE 1: Classic Processing ----------------------

## Generate data
set.seed(100)
n <- 350
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE)
x[25] <- 200
x[320] <- 170
df <- data.frame(timestamp = 1:n, value = x)

## Calculate anomalies
result <- IpSdEwma(
  data = df$value,
  n.train = 5,
  threshold = 0.01,
  l = 2
)
res <- cbind(df, result$result)

## Plot results
PlotDetections(res, title = "SD-EWMA ANOMALY DETECTOR")

## Reduce anomalies
res$is.anomaly <- ReduceAnomalies(res$is.anomaly, windowLength = 5)

## Plot results
PlotDetections(res, title = "SD-EWMA ANOMALY DETECTOR")


## EXAMPLE 2: Incremental Processing ----------------------


  # install.packages("stream")
  library("stream")

  # Generate data
  set.seed(100)
  n <- 350
  x <- sample(1:100, n, replace = TRUE)
  x[70:90] <- sample(110:115, 21, replace = TRUE)
  x[25] <- 200
  x[320] <- 170
  df <- data.frame(timestamp = 1:n, value = x)
  dsd_df <- DSD_Memory(df)

  # Initialize parameters for the loop
  last.res <- NULL
  red.res <- NULL
  res <- NULL
  nread <- 100
  numIter <- ceiling(n/nread)

  # Calculate anomalies
  for(i in 1:numIter) {
    # read new data
    newRow <- get_points(dsd_df, n = nread, outofpoints = "ignore")
    # calculate if it's an anomaly
    last.res <- IpSdEwma(
      data = newRow$value,
      n.train = 5,
      threshold = 0.01,
      l = 2,
      last.res = last.res$last.res
    )

    if(!is.null(last.res$result)){
      # reduce anomalies
      red.res <- ReduceAnomalies(last.res$result$is.anomaly,
                                 windowLength = 5, incremental = TRUE, last.res = red.res$last.res)
      last.res$result$is.anomaly <- red.res$result

      # prepare the result
      res <- rbind(res, cbind(newRow, last.res$result))
    }
  }

  # Plot results
  PlotDetections(res, title = "SD-EWMA ANOMALY DETECTOR")

alaineiturria/otsad documentation built on Jan. 12, 2023, 12:26 p.m.