working with weirds"

knitr::opts_chunk$set(echo = TRUE,  
                      fig.width=10 ,
                      fig.height=7,
                      out.width='100%',
                      dpi=100)
library(knitr)
library(dplyr)
library(ggplot2)
suppressMessages(library(stranger)) 
library(tidyr)

Anomalies manual selection

For some metrics, litterature recommends some cutpoints (2 for lof for instance).

There is a way to derive anomalies from this given cutpoint by filtering a stranger or a singularize object and convert to anomalies the set of retained records.

Filtering can be done either with usual [ or dplyr select.

Assumed dbscan package is installed, one can for instance use lof weird, use 2 as cutpoint and derive an anomaly object with as.anomalies.

iris %>% select(-Species) %>%
  crazyfy() %>%
  strange(weird="lof") %>%
  singularize() -> s
anom <- filter(s,lof_k_10>2) %>% as.anomalies()

NOTE our current implementation of select is not pipeable; we are still trying to ingest Hadley Wickham litterature on programming with dyplyr...

Anomaly objects are simple vectors with attributes; once can then access the first anomaly and display it with:

plot(s,type="neighbours", score="lof_k_10",anomaly_id=anom[1])

Using a tuneGrid

stranger package is a wrapper around several methods. Problem thus arrives on the choice of the method... We will see later in this vignette how to work with several metrics and in particular the stacking.

But world is really complex and even for one chosen method, leyt say knn weird, user has to pick some parameters...

Here comes the parameter tuneGrid (name borrowed from caret) that allow to use one weird with different sets of parameters in the same call.

Let's prepare a matrix of parameters:

tg <- expand.grid(k=c(5,10,20),simplify=c("mean","median"),stringsAsFactors=FALSE)
tg

We thus have 6 possible combinations

(anoms.all <-  iris %>% select(-Species) %>%
  crazyfy() %>% strange(weird="knn",tuneGrid=tg))

NOTES

tg = data.frame(k=c(5,5:8))
(anoms <-  iris %>% select(-Species) %>%
  crazyfy() %>% strange(weird="knn",tuneGrid=tg,algorithm=c("cover_tree")))
(meta  <- get_info(anoms))
meta[,"parameters"]

Merging stranger objects

So you have decided to go for and try two different methods. Let say knn and autoencode.

First, we can create two object containg assciatdd metrics.

data <- iris %>% select(-Species) %>% crazyfy()
m1 <- strange(data, weird="knn")
m2 <- strange(data, weird="autoencode")

For convenience and further exploitation, a merge method is at your disposal to gather the two corresponding metrics in a single object:

(metrics <- merge(m1,m2))

Invoke several weirds at once (stranger)

You are not sure yet about the metric you want to use. Or you would like to test many, with plenty of different values for the parameters? stranger is for you. This function is similar to caretList in caretEnsemble package.

With its first parameter methodList you can supply many methods that will be invoked with their default parameter.

If you want to use your own values or use a tuneGrid for a given method, you will have to create weird objects using weird function and pass a list of such weirds object to the parameter tuneList.

Following call will for instance: fit a knn weird with default values (taken from methodList) fit a autoencode weird with default values * also fit a knn but with k set to 20 (not recommended for iris data having 50 observations)

s3=stranger(data,
    methodList=c("knn","autoencode"),
    tuneList=list(weird(method="knn",k=20)))

does not work (autoencode tuneGrid)

ss=stranger(data, methodList=c("lof"), tuneList=list( weird(method="knn", tuneGrid=data.frame(k=c(3,5,10,15))) , weird(method="autoencode",colname="auto_2hidden",nl=4,N.hidden=c(6,15)) , weird(method="autoencode", colname="auto", tuneGrid=data.frame(nl=3,N.hidden=c(6,15,30)))))

anom <- ss %>% singularize() %>% get_anomalies(nmin=6,nmax=6)

anom %>% fortify(data=iris,colname="anomaly") %>% ggplot(aes(x=Sepal.Length,y=Sepal.Width,color=Species,size=anomaly))+geom_point()+scale_size_discrete(range=c(1,3))

Normalize & stack metrics



Try the stranger package in your browser

Any scripts or data that you put into this service are public.

stranger documentation built on March 18, 2018, 2:01 p.m.