Data complexity based filters for removing label noise from a dataset as a preprocessing step of classification. For more information, see 'Details' and 'References' sections.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
## S3 method for class 'formula' saturationFilter(formula, data, ...) ## Default S3 method: saturationFilter(x, noiseThreshold = NULL, classColumn = ncol(x), ...) ## S3 method for class 'formula' consensusSF(formula, data, ...) ## Default S3 method: consensusSF(x, nfolds = 10, consensusLevel = nfolds - 1, noiseThreshold = NULL, classColumn = ncol(x), ...) ## S3 method for class 'formula' classifSF(formula, data, ...) ## Default S3 method: classifSF(x, nfolds = 10, noiseThreshold = NULL, classColumn = ncol(x), ...)
A formula describing the classification variable and the attributes to be used.
Data frame containing the tranining dataset to be filtered.
Optional parameters to be passed to other methods.
The threshold for removing noisy instances in the saturation filter.
Authors recommend values between 0.25 and 2. If it is set to
Positive integer indicating the column which contains the (factor of) classes. By default, the last column is considered.
Based on theoretical studies about data complexity (Gamberger & Lavrac, 1997),
saturationFilter removes those
instances which most enable to reduce the CLCH (Complexity of the Least Complex Hypotheses)
of the training dataset. The full method can be looked up in (Gamberger et al., 1999), and
the previous step of literals extraction is detailed in (Gamberger et al., 1996).
consensusSF splits the dataset in
nfolds folds, and applies
saturationFilter to every combination of
nfolds-1 folds. Those instances
with (at least)
consensusLevel 'noisy votes' are removed.
saturationFilter with a
nfolds-folds cross validation
scheme (the latter in the spirit of filters such as
Namely, the dataset is split in
nfolds folds and, for every combination
saturationFilter is applied and a classifier
(we implement a standard C4.5 tree) is built. Instances
from the excluded fold are removed according to this classifier.
An object of class
filter, which is a list with seven components:
cleanData is a data frame containing the filtered dataset.
remIdx is a vector of integers indicating the indexes for
removed instances (i.e. their row number with respect to the original data frame).
repIdx is a vector of integers indicating the indexes for
repaired/relabelled instances (i.e. their row number with respect to the original data frame).
repLab is a factor containing the new labels for repaired instances.
parameters is a list containing the argument values.
call contains the original call to the filter.
extraInf is a character that includes additional interesting
information not covered by previous items.
Gamberger D., Lavrac N., Groselj C. (1999, June): Experiments with noise filtering in a medical domain. In ICML (pp. 143-151).
Gamberger D., Lavrac N., Dzeroski S. (1996, January): Noise elimination in inductive concept learning: A case study in medical diagnosis. In Algorithmic Learning Theory (pp. 199-212). Springer Berlin Heidelberg.
Gamberger D., Lavrac N. (1997): Conditions for Occam's razor applicability and noise elimination (pp. 108-123). Springer Berlin Heidelberg.
1 2 3 4 5 6 7 8 9 10 11