LocScaleB: Univariate outlier detection with bounds based on robust...

In univOutl: Detection of Univariate Outliers

Univariate outlier detection with bounds based on robust location and scale estimates

Description

This function identifies outliers in the tails of a distribution by detecting the observations outside the bounds built using a robust estimate of both location and scale parameters.

Usage

LocScaleB(x, k=3, method='MAD',  weights=NULL, id=NULL, 
          exclude=NA, logt=FALSE, return.dataframe=FALSE)

Arguments

x	Numeric vector that will be searched for outliers.
k	Nonnegative constant that determines the extension of bounds. Commonly used values are 2, 2.5 and 3 (default).
method	character identifying how to estimate the scale of the distribution. Available choices are: method='IQR' for using the Inter-Quartile Range, i.e. Q3-Q1; method='IDR' for using the Inter-Decile Range; i.e. P90-P10 method='MAD' for using the Median Absolute Deviation; method='Gini' robust scale estimate based on Gini's Mean Difference (see GiniMd); method='ScaleTau2' robust tau-estimate of univariate scale, as proposed by Maronna and Zamar (2002) (see alsoscaleTau2); method='Qn' for using the Qn estimator proposed by Rousseeuw and Croux (1993) (see also Qn); method='Sn' for using the Sn estimator proposed by Rousseeuw and Croux (1993) (see also Sn). When method='dQ' the estimated scale for the left tail is (Q2-Q1)/0.6745, while for the right tail it is considered (Q3-Q2)/0.6745 (Q2 is the median); this double estimate should be able to account for slight skewness. When method='dD' the estimated scale for the left tail is (P50-P10)/1.2816, while for the right tail it is considered (P90-P50)/1.2816 (P50 is the median); this double estimate should be able to account for skewness. Finally, when method='AdjOut', bounds are based on the adjusted outlyingness method as proposed by Hubert and Van der Veeken (2008).
weights	Optional numeric vector that provides weights associated to observations. Only nonnegative weights are allowed. Note that weights can only be used when method='MAD', method='IQR', method='IDR', method='dQ' or method='dD'.
id	Optional numeric or character vector, with identifiers of units in x. If id=NULL (default) units' identifiers will be set equal to their position in x.
exclude	Values of x that will be excluded by the analysis. By default missing values (exclude = NA)
logt	Logical, if TRUE, before searching outliers the x variable is log-transformed (log(x+1) is considered). Note that in this case that summary output (bounds, etc.) will refer to log-transformed variable.
return.dataframe	Logical, if TRUE the output will save all the relevant information for outlier detection in a dataframe with the following columns: ‘id’ (units' identifiers), ‘x’, ‘log.x’ (only if logt=TRUE), ‘weight’ (only when argument weights is provided), ‘score’ (the standardized scores, see Details) and, finally, ‘outliers’, where value 1 indicates observations detected as an outlier, 0 otherwise.

Details

The intervals are derived by considering the median Q_2 as a robust location estimate while different robust scale estimators are considered:

[Q2 - k*s_L; Q2 + k*s_R]

where s_L and S_R are robust scale estimates. With most of the methods s_L=s_L with exception of method='dQ' and method='dD' where respectively:

s_L = (Q2 - Q1)/0.6745 and s_R = (Q3 - Q2)/0.6745

and

s_L = (P50 - P10)/1.2816 and s_R = (P90 - P50)/1.2816

Note that when method='dQ' or method='dD' the function calculates and prints a the Bowley's coefficient of skewness, that uses Q1, Q2 and Q3 (they are replaced by respectively P10, P50 and P90 when method='dD').

With method='AdjOut' the following estimates are considered:

s_L = (Q2 - fL) ands_R = (fR - Q2)

being fR and fL derived starting from the fences of the adjusted boxplot (Hubert and Vandervieren, 2008; see adjboxStats). In addition the medcouple (mc) measure of skewness is calculated and printed on the screen.

When weights are available (passed via the argument weights) then they are used in the computation of the quartiles. In particular, the quartiles are derived using the function wtd.quantile in the package Hmisc. Note that their use is allowed just with method='IQR', method='IDR', method='dQ', method='dD' or method='AdjOut'.

The ‘score’ variable reported in the the data dataframe when return.dataframe=TRUE is the standardized score derived as (x - Median)/scale.

Value

A list whose components depend on the return.dataframe argument. When return.dataframe = FALSE just the following components are provided:

pars	Vector with estimated median and scale parameters
bounds	The bounds of the interval, values outside the interval are considered outliers.
excluded	The position or identifiers of x values excluded by outlier detection, according to the argument exclude
outliers	The position or identifiers of x values detected as outliers (outside bounds).
lowOutl	The identifiers or positions (when id=NULL) of units in x detected as outliers in the lower tail of the distribution.
upOutl	The identifiers or positions (when id=NULL) of units in x detected as outliers in the upper tail of the distribution.

When return.dataframe=TRUE the latter two components are substituted with two dataframes:

excluded	A dataframe with the subset of observations excluded.
data	A dataframe with the the not excluded observations and the following columns: ‘id’ (units' identifiers), ‘x’, ‘log.x’ (only if logt=TRUE), ‘weight’ (only when argument weights is provided), ‘score’ (the standardized scores, see Details) and, finally, ‘outliers’, where value 1 indicates observations detected as an outlier and 0 otherwise.

Author(s)

Marcello D'Orazio [e-mail]

References

Hubert, M. and Van der Veeken, S. (2008) ‘Outlier Detection for Skewed Data’. Journal of Chemometrics, 22, pp. 235-246.

Maronna, R.A. and Zamar, R.H. (2002) ‘Robust estimates of location and dispersion of high-dimensional datasets’ Technometrics, 44, pp. 307-317.

Rousseeuw, P.J. and Croux, C. (1993) ‘Alternatives to the Median Absolute Deviation’, Journal of the American Statistical Association 88, pp. 1273-1283.

Vanderviere, E. and Huber, M. (2008) ‘An Adjusted Boxplot for Skewed Distributions’, Computational Statistics & Data Analysis, 52, pp. 5186-5201

See Also

mad, scaleTau2, Qn, Sn, GiniMd

Examples

set.seed(333)
x <- rnorm(30, 50, 1)
x[10] <- 1
x[20] <- 100

out <- LocScaleB(x = x,  k = 3, method='MAD')
out$pars
out$bounds
out$outliers
x[out$outliers]

out <- LocScaleB(x = x,  k = 3, method='MAD',
                 return.dataframe = TRUE)
head(out$data)

out <- LocScaleB(x = x, k = 3, method='AdjOut')
out$outliers

univOutl documentation built on May 31, 2022, 9:06 a.m.