GetDetectorScore: Get detector score

In alaineiturria/otsad: Online Time Series Anomaly Detectors

Get detector score

Description

GetDetectorScore Calculates the start and end positions of each window that are focused on the real anomalies. This windows can be used to know if the detected anomaly is a true positive or not.

Usage

GetDetectorScore(data, print = FALSE, title = "")

Arguments

data	All dataset with training and test datasets and with at least timestamp, value, is.anomaly and is.real.anomaly columns.
print	If TRUE shows a graph with results.
title	Title of the graph.

Details

data must be a data.frame with timestamp, value, is.anomaly and is.real.anomaly columns. timestamp column can be numeric, of type POSIXct, or a character type date convertible to POSIXct.

This function calculates the scores based on three different profiles. Each label tp, fp, tn, fn is associated with a weight to give a more realistic score. For the standard profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 1. For the reward_low_FP_rate profile weights are tp = 1, tn = 1, fp, = 0.22, and fn = 1. For the reward_low_FN_rate profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 2.

Value

List conformed by the following items:

data	Same data set with additional columns such as label, start.limit, end.limit, standard.score and etc.
standard	Total score obtained by the detector using the weights of the standard profile.
low_FP_rate	Total score obtained by the detector using the weights of the reward_low_FP_rate profile.
low_FN_rate	Total score obtained by the detector using the weights of the reward_low_FN_rate profile.
tp	Number of true positives
tn	Number of true negatives
fp	Number of false positives
fn	Number of false negatives

References

A. Lavin and S. Ahmad, “Evaluating Real-time Anomaly Detection Algorithms – the Numenta Anomaly Benchmark,” in 14th International Conference on Machine Learning and Applications (IEEE ICMLA’15), 2015.

Examples

## Generate data
set.seed(100)
n <- 180
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE)
x[25] <- 200
x[150] <- 170
df <- data.frame(timestamp = 1:n, value = x)

# Add is.real.anomaly column
df$is.real.anomaly <- 0
df[c(25,80,150), "is.real.anomaly"] <- 1

## Calculate anomalies
result <- CpSdEwma(
  data = df$value,
  n.train = 5,
  threshold = 0.01,
  l = 3
)
res <- cbind(df, result)

# Get detector score
GetDetectorScore(res, print = FALSE, title = "")

alaineiturria/otsad documentation built on Jan. 12, 2023, 12:26 p.m.