# covNNC: Robust Covariance Estimation via Nearest Neighbor Cleaning In robustX: 'eXtra' / 'eXperimental' Functionality for Robust Statistics

## Description

`covNNC()` estimates robust covariance/dispersion matrices by the nearest neighbor variance estimation (NNVE) or (rather) “Nearest Neighbor Cleaning” (NNC) method of Wang and Raftery (2002, JASA).

## Usage

 ```1 2``` ```covNNC(X, k = min(12, n - 1), pnoise = 0.05, emconv = 0.001, bound = 1.5, extension = TRUE, devsm = 0.01) ```

## Arguments

 `X` matrix in which each row represents an observation or point and each column represents a variable. `k` desired number of nearest neighbors (default is 12) `pnoise` percent of added noise `emconv` convergence tolerance for EM `bound` value used to identify surges in variance caused by outliers wrongly included as signal points (`bound = 1.5` means a 50 percent increase) `extension` whether or not to continue after reaching the last chi-square distance. The default is to continue, which is indicated by setting `extension = TRUE`. `devsm` when `extension = TRUE`, the algorithm stops if the relative difference in variance is less than `devsm`. (default is 0.01)

## Value

A list with components

 `cov` covariance matrix `mu` mean vector `postprob` posterior probability `classification` classification (0=noise otherwise 1) obtained by rounding `postprob` `innc` list of initial nearest neighbor cleaning results (components are the covariance, mean, posterior probability and classification)

## Note

MM: Even though `covNNC()` is backed by a serious scientific publication, I cannot recommend its use at all.

## Author(s)

Naisyin Wang [email protected] and Adrian Raftery [email protected] with contributions from Chris Fraley [email protected].

`covNNC()`, then named `cov.nnve()`, used to be (the only function) in CRAN package covRobust (2003), which was archived in 2012.

Martin Maechler allowed `ncol(X) == 1`, sped up the original code, by reducing the amount of scaling; further, the accuracy was increased (using internal `q.dDk()`). The original version is available, unexported as `robustX:::covNNC1`.

## References

Wang, N. and Raftery, A. (2002) Nearest neighbor variance estimation (NNVE): Robust covariance estimation via nearest neighbor cleaning (with discussion). Journal of the American Statistical Association 97, 994–1019.

`cov.mcd` from package MASS; `covMcd`, and `covOGK` from package robustbase.

The whole package rrcov.

## Examples

 ```1 2 3 4 5 6``` ```data(iris) covNNC(iris[-5]) data(hbk, package="robustbase") hbk.x <- data.matrix(hbk[, 1:3]) covNNC(hbk.x) ```

### Example output

```\$cov
Sepal.Length Sepal.Width Petal.Length Petal.Width
Sepal.Length   0.58766332 -0.07398017    1.1541669   0.4825507
Sepal.Width   -0.07398017  0.13837244   -0.3417718  -0.1296508
Petal.Length   1.15416691 -0.34177177    2.9542649   1.2567524
Petal.Width    0.48255066 -0.12965076    1.2567524   0.5754763

\$mu
Sepal.Length  Sepal.Width Petal.Length  Petal.Width
5.795426     3.071265     3.680866     1.176461

\$postprob
[1]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[6]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[11]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  9.982665e-01
[16] 5.660588e-158  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[21]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[26]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[31]  1.000000e+00  1.000000e+00  9.999986e-01  1.063175e-24  1.000000e+00
[36]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[41]  1.000000e+00 3.163497e-136  1.000000e+00  1.000000e+00  1.000000e+00
[46]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[51]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[56]  1.000000e+00  1.000000e+00  9.999852e-01  1.000000e+00  1.000000e+00
[61]  4.951128e-74  1.000000e+00  4.039360e-05  1.000000e+00  1.000000e+00
[66]  1.000000e+00  1.000000e+00  1.000000e+00  1.039353e-04  1.000000e+00
[71]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[76]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[81]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[86]  9.998382e-01  1.000000e+00  9.999911e-01  1.000000e+00  1.000000e+00
[91]  1.000000e+00  1.000000e+00  1.000000e+00  5.615524e-04  1.000000e+00
[96]  1.000000e+00  1.000000e+00  1.000000e+00  9.999995e-01  1.000000e+00
[101]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[106]  9.976415e-01  3.168355e-01  9.999997e-01  9.988788e-01  4.922098e-37
[111]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[116]  1.000000e+00  1.000000e+00 3.122314e-234 8.376891e-121  4.733103e-07
[121]  1.000000e+00  1.000000e+00  7.047000e-49  1.000000e+00  1.000000e+00
[126]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[131]  3.573661e-01 4.213386e-292  1.000000e+00  1.000000e+00  1.000000e+00
[136]  9.991833e-01  9.999999e-01  1.000000e+00  1.000000e+00  1.000000e+00
[141]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[146]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00

\$classification
[1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1
[38] 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 0 1 1 1 1 1
[75] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1
[112] 1 1 1 1 1 1 0 0 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[149] 1 1

\$iter
[1] 5

\$innc
\$innc\$cov
Sepal.Length Sepal.Width Petal.Length Petal.Width
Sepal.Length   0.53013518 -0.08128338    1.0968311   0.4751778
Sepal.Width   -0.08128338  0.10970763   -0.3225952  -0.1229924
Petal.Length   1.09683108 -0.32259520    2.8234407   1.2164172
Petal.Width    0.47517780 -0.12299238    1.2164172   0.5593988

\$innc\$mu
Sepal.Length  Sepal.Width Petal.Length  Petal.Width
5.723749     3.062876     3.560977     1.127211

\$innc\$iter
[1] 4

\$innc\$postprob
[1]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[6]  1.711343e-01  1.000000e+00  1.000000e+00  9.999997e-01  1.000000e+00
[11]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  2.309910e-19
[16] 7.747016e-268  7.177075e-01  1.000000e+00  1.963346e-03  1.000000e+00
[21]  1.000000e+00  1.000000e+00  9.999999e-01  1.000000e+00  1.000000e+00
[26]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[31]  1.000000e+00  1.000000e+00  1.441755e-14  7.063285e-61  1.000000e+00
[36]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[41]  1.000000e+00  0.000000e+00  1.000000e+00  1.000000e+00  9.999874e-01
[46]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[51]  5.812702e-02  1.000000e+00  1.000000e+00  7.726717e-01  1.000000e+00
[56]  1.000000e+00  9.994284e-01  3.762373e-16  1.000000e+00  9.999999e-01
[61] 1.745179e-137  1.000000e+00  1.795669e-30  1.000000e+00  1.000000e+00
[66]  9.999987e-01  1.000000e+00  1.000000e+00  7.792111e-30  1.000000e+00
[71]  9.999874e-01  1.000000e+00  9.999570e-01  1.000000e+00  1.000000e+00
[76]  1.000000e+00  9.999996e-01  1.000000e+00  1.000000e+00  9.999999e-01
[81]  9.999664e-01  9.989320e-01  1.000000e+00  1.000000e+00  1.000000e+00
[86]  9.202298e-18  1.000000e+00  8.346147e-16  1.000000e+00  9.999999e-01
[91]  1.000000e+00  1.000000e+00  1.000000e+00  1.070669e-28  1.000000e+00
[96]  1.000000e+00  1.000000e+00  1.000000e+00  7.200729e-14  1.000000e+00
[101]  8.234718e-08  9.999999e-01  1.000000e+00  1.000000e+00  1.000000e+00
[106]  1.430659e-19  3.621917e-24  1.406446e-13  4.549045e-19  4.953561e-80
[111]  1.000000e+00  1.000000e+00  1.000000e+00  1.189762e-05  8.985381e-01
[116]  1.000000e+00  1.000000e+00  0.000000e+00 4.078622e-210  1.800131e-33
[121]  1.000000e+00  9.999987e-01  2.005512e-98  1.000000e+00  1.000000e+00
[126]  2.294380e-01  1.000000e+00  1.000000e+00  9.999998e-01  7.022288e-01
[131]  4.801429e-24  0.000000e+00  9.999874e-01  1.000000e+00  8.176383e-01
[136]  7.444583e-19  1.195883e-12  1.000000e+00  1.000000e+00  1.000000e+00
[141]  1.000000e+00  1.000000e+00  9.999999e-01  1.000000e+00  9.998421e-01
[146]  1.000000e+00  9.914360e-01  1.000000e+00  2.058725e-04  1.000000e+00

\$innc\$classification
[1] 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1
[38] 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1 0 1 1 1 1 1
[75] 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 0 1 1 1 1 0 1 0 1 1 1 1 0 0 0 0 0 1
[112] 1 1 0 1 1 1 0 0 0 1 1 0 1 1 0 1 1 1 1 0 0 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1
[149] 0 1

\$cov
X1         X2        X3
X1 1.11349637 0.04992206 0.1154125
X2 0.04992206 1.13338350 0.1384305
X3 0.11541252 0.13843053 1.0526149

\$mu
X1       X2       X3
1.537705 1.780328 1.686885

\$postprob
[1]  0.000000e+00  0.000000e+00  0.000000e+00 4.979277e-194 1.776498e-264
[6]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00
[11]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  1.000000e+00
[16]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[21]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[26]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[31]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[36]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[41]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[46]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[51]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[56]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[61]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[66]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[71]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00

\$classification
[1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[39] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

\$iter
[1] 2

\$innc
\$innc\$cov
X1         X2        X3
X1 1.11349637 0.04992206 0.1154125
X2 0.04992206 1.13338350 0.1384305
X3 0.11541252 0.13843053 1.0526149

\$innc\$mu
X1       X2       X3
1.537705 1.780328 1.686885

\$innc\$iter
[1] 2

\$innc\$postprob
[1]  0.000000e+00  0.000000e+00  0.000000e+00 2.470657e-193 7.355258e-264
[6]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00
[11]  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  1.000000e+00
[16]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[21]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[26]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[31]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[36]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[41]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[46]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[51]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[56]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[61]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[66]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00
[71]  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00  1.000000e+00

\$innc\$classification
[1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
[39] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
```

robustX documentation built on May 2, 2019, 5:16 p.m.