condextr: Remove outliers using point-by-point weighed outlier removal...

In dataprep: Fast, Efficient, and Versatile Data Preprocessing and Reshaping with 'C++', 'OpenMP' & 'SIMD'

Remove outliers using point-by-point weighed outlier removal by conditional extremum

Description

An outlier removal method that considers each potential outlier individually, using conditional thresholds based on top and bottom percentiles and error margins. It is a more refined alternative to simple percentile-based removal.

Usage

condextr(data, cols = NULL, group = NULL, top = 0.995,
top.error = 0.1, top.magnitude = 0.2, bottom = 0.0025, bottom.error = 0.2,
bottom.magnitude = 0.4, interval = 10, by = "min", half = 30,
times = 10, date_col = NULL, cores = NULL, verbose = FALSE)

Arguments

data	A data frame, matrix, or numeric vector. If a vector, outlier removal is applied directly.
cols	Column indices or names of numeric variables.
group	Grouping column index or name. Outlier removal is performed within each group.
top, top.error, top.magnitude	Parameters for the top threshold.
bottom, bottom.error, bottom.magnitude	Parameters for the bottom threshold.
interval	Number of outlier marking iterations between observation deletions.
by	Time unit for observation deletion. An invalid unit string raises an error.
half	Half window size in minutes for observation deletion.
times	Number of observation deletions.
date_col	Time column index or name.
cores	Number of CPU cores passed to the C++ backend. NULL (default) lets the backend choose based on data size.
verbose	Logical; if TRUE, prints timing and deletion counts.

Details

The method marks values as outliers if they are the maximum or minimum and exceed a threshold that depends on a high percentile plus an error margin. After a specified number of marking steps, observation deletion is applied to remove rows with excessive missing values created by outlier removal.

Value

A data frame with outliers removed.

Author(s)

Chun-Sheng Liang <[e-mail]>

References

1. Example data is from https://smear.avaa.csc.fi/download. It includes particle number concentrations in SMEAR I Varrio forest.

Examples

condextr(obsedele(data[1:500, c(1, 4, 17:19)], cols = 3:5, group = 2),
         cols = 3:5, group = 2)

dataprep documentation built on Oct. 1, 2026, 5:07 p.m.