GetNullAndPerfectScores: Get Null And Perfect Scores

In otsad: Online Time Series Anomaly Detectors

Description

GetNullAndPerfectScores Calculates the score of Perfect and Null detectors scores. Perfect detector is one that outputs all true positives and no false positives. And Null detector is one that outputs no anomaly detections.

Usage

GetNullAndPerfectScores(data)

Arguments

data	All dataset with training and test datasets and with at least timestamp, value and is.real.anomaly columns.

Details

This function calculates the scores based on three different profiles. Each tp, fp, tn, fn label is associated with a weight to give a more realistic score. For the standard profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 1. For the reward_low_FP_rate profile weights are tp = 1, tn = 1, fp, = 0.22, and fn = 1. For the reward_low_FN_rate profile weights are tp = 1, tn = 1, fp, = 0.11, and fn = 2.

Value

data.frame with null and perfect detectors scores for each profile.

References

A. Lavin and S. Ahmad, “Evaluating Real-time Anomaly Detection Algorithms – the Numenta Anomaly Benchmark,” in 14th International Conference on Machine Learning and Applications (IEEE ICMLA’15), 2015.

Examples

## Generate data
set.seed(100)
n <- 180
x <- sample(1:100, n, replace = TRUE)
x[70:90] <- sample(110:115, 21, replace = TRUE)
x[25] <- 200
x[150] <- 170
df <- data.frame(timestamp = 1:n, value = x)

# Add is.real.anomaly column
df$is.real.anomaly <- 0
df[c(25,80,150), "is.real.anomaly"] <- 1

# Get null and perfect scores
GetNullAndPerfectScores(df)

otsad documentation built on Sept. 6, 2019, 5:02 p.m.