View source: R/directional_outlyingness.R
| directional_outlyingness | R Documentation |
This function computes the directional outlyingness of a numeric vector, as proposed by Rousseeuw et al. (2018), which is a measure of outlyingness of data point that takes the skewness of the underlying distribution into account.
directional_outlyingness(
x,
cutoff_quantile = 0.995,
rm_zeroes = FALSE,
max_ratio = NULL,
prec_scale = 1e-10,
cutoff.quantile = deprecated(),
rmZeroes = deprecated(),
maxRatio = deprecated(),
precScale = deprecated()
)
x |
A numeric vector |
cutoff_quantile |
A numeric value in (0, 1): the proportion of the
observations of a normal distribution that are not flagged. The
cutoff of the scores is the |
rm_zeroes |
A logical value. If |
max_ratio |
|
prec_scale |
A non-negative number: an initial scale below it is taken as zero (default: 1e-10). |
cutoff.quantile, rmZeroes, maxRatio, precScale |
Directional outlyingness takes the potential skewness of the underlying
distribution into account, by splitting the univariate dataset in two half
samples around the median, each with
h = \lfloor (n+1)/2 \rfloor
observations (they share the median when n is odd), and scaling each
half separately (Rousseeuw et al., 2018, Section 2.1). For the distances
z_i of a half sample to the median, the scale is the one-step
M-estimator
s = s_0 \sqrt{\frac{1}{2 \alpha h} \sum_i \rho_c\left(\frac{z_i}{s_0}\right)},
from the initial scale
s_0 = \mathrm{med}_i(z_i) / \Phi^{-1}(0.75),
with Huber's \rho-function for scale,
\rho_c(t) = \min\{(t/c)^2, 1\},
c = 2.1 (85% efficiency at
the normal distribution), and
\alpha = \int_0^\infty \rho_c(x) \, d\Phi(x), which makes the
scale consistent at the normal distribution.
The score of an observation is its distance to the median divided by the
scale of its own half: on the short side of a skewed distribution, an
observation gets a higher score than one as far from the median on the
long side. For a normal distribution, both scales estimate the standard
deviation, and the score is close to the absolute robust z-score of
zscore().
An observation is flagged when its score exceeds the
(1 + q) / 2 quantile of the standard normal distribution, where
q is cutoff_quantile (2.81 for the default 0.995): about
1 - q of the observations of a normal distribution (0.5%) are
flagged.
A tibble with columns:
data: The original numeric values.
score: The calculated outlyingness score.
flag: A logical vector indicating whether each value is a potential outlier or not.
Christian L. Goueguel
Rousseeuw, P.J., Raymaekers, J., Hubert, M., (2018). A Measure of Directional Outlyingness With Applications to Image Data and Video. Journal of Computational and Graphical Statistics, 27(2):345–359.
zscore(), iqr_outliers()
# 200 right-skewed values (a gamma distribution shifted to start at 100),
# 3 low outliers below them and 3 high outliers
set.seed(1)
x <- c(100 + rgamma(200, shape = 2, scale = 10), 85, 87, 89, 220, 230, 240)
group <- rep(c("regular", "outlier"), c(200, 6))
ggplot2::ggplot(data.frame(x, group), ggplot2::aes(x)) +
ggplot2::geom_density(fill = "#2166ac", alpha = 0.3) +
ggplot2::geom_rug(ggplot2::aes(colour = group)) +
ggplot2::scale_colour_manual(values = c(outlier = "#b2182b", regular = "#2166ac"))
# the robust z-score scales both sides alike: regular values of the long
# right tail are flagged, but not the low outliers (the last three rows)
rob <- zscore(x, robust = TRUE)
subset(rob, flag)
tail(rob, 3)
# directional outlyingness scales each side separately: the low outliers
# score above the regular values of the long tail, and the 6 outliers are
# flagged
head(directional_outlyingness(x), 8)
data(forageLIBS)
# iron contents (mg/kg): a few samples are far above the others
fe <- forageLIBS$Fe[!is.na(forageLIBS$Fe)]
head(directional_outlyingness(fe))
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.