ensemble_fs_result: Ensemble Feature Selection Result

In mlr3fselect: Feature Selection for 'mlr3'

Ensemble Feature Selection Result

Description

The EnsembleFSResult stores the results of ensemble feature selection. It includes methods for evaluating the stability of the feature selection process and for ranking the selected features among others. The function ensemble_fselect() returns an object of this class.

S3 Methods

• as.data.table.EnsembleFSResult(x, benchmark_result = TRUE)
  Returns a tabular view of the ensemble feature selection.
  EnsembleFSResult -> data.table::data.table()
  

  • x (EnsembleFSResult)

  • benchmark_result (logical(1))
    Whether to add the learner, task and resampling information from the benchmark result.

Public fields

benchmark_result

(mlr3::BenchmarkResult)
The benchmark result.

man

(character(1))
Manual page for this object.

Active bindings

result

(data.table::data.table)
Returns the result of the ensemble feature selection.

n_learners

(numeric(1))
Returns the number of learners used in the ensemble feature selection.

measure

(character(1))
Returns the measure id used in the ensemble feature selection.

Methods

Public methods

• EnsembleFSResult$new()

• EnsembleFSResult$format()

• EnsembleFSResult$print()

• EnsembleFSResult$help()

• EnsembleFSResult$feature_ranking()

• EnsembleFSResult$stability()

• EnsembleFSResult$pareto_front()

• EnsembleFSResult$knee_points()

• EnsembleFSResult$clone()

---

Method new()

Creates a new instance of this R6 class.

Usage

EnsembleFSResult$new(
  result,
  features,
  benchmark_result = NULL,
  measure_id,
  minimize = TRUE
)

Arguments

result

(data.table::data.table)
The result of the ensemble feature selection. Column names should include "resampling_iteration", "learner_id", "features" and "n_features".

features

(character())
The vector of features of the task that was used in the ensemble feature selection.

benchmark_result

(mlr3::BenchmarkResult)
The benchmark result object.

measure_id

(character(1))
Column name of "result" that corresponds to the measure used.

minimize

(logical(1))
If TRUE (default), lower values of the measure correspond to higher performance.

---

Method format()

Helper for print outputs.

Usage

EnsembleFSResult$format(...)

Arguments

...

(ignored).

---

Method print()

Printer.

Usage

EnsembleFSResult$print(...)

Arguments

...

(ignored).

---

Method help()

Opens the corresponding help page referenced by field ⁠$man⁠.

Usage

EnsembleFSResult$help()

---

Method feature_ranking()

Calculates the feature ranking.

Usage

EnsembleFSResult$feature_ranking(method = "approval_voting")

Arguments

method

(character(1))
The method to calculate the feature ranking.

Details

The feature ranking process is built on the following framework: models act as voters, features act as candidates, and voters select certain candidates (features). The primary objective is to compile these selections into a consensus ranked list of features, effectively forming a committee. Currently, only "approval_voting" method is supported, which selects the candidates/features that have the highest approval score or selection frequency, i.e. appear the most often.

Returns

A data.table::data.table listing all the features, ordered by decreasing inclusion probability scores (depending on the method)

---

Method stability()

Calculates the stability of the selected features with the stabm package. The results are cached. When the same stability measure is requested again with different arguments, the cache must be reset.

Usage

EnsembleFSResult$stability(
  stability_measure = "jaccard",
  stability_args = NULL,
  global = TRUE,
  reset_cache = FALSE
)

Arguments

stability_measure

(character(1))
The stability measure to be used. One of the measures returned by stabm::listStabilityMeasures() in lower case. Default is "jaccard".

stability_args

(list)
Additional arguments passed to the stability measure function.

global

(logical(1))
Whether to calculate the stability globally or for each learner.

reset_cache

(logical(1))
If TRUE, the cached results are ignored.

Returns

A numeric() value representing the stability of the selected features. Or a numeric() vector with the stability of the selected features for each learner.

---

Method pareto_front()

This function identifies the Pareto front of the ensemble feature selection process, i.e., the set of points that represent the trade-off between the number of features and performance (e.g. classification error).

Usage

EnsembleFSResult$pareto_front(type = "empirical")

Arguments

type

(character(1))
Specifies the type of Pareto front to return. See details.

Details

Two options are available for the Pareto front:

• "empirical" (default): returns the empirical Pareto front.

• "estimated": the Pareto front points are estimated by fitting a linear model with the inversed of the number of features (1/x) as input and the associated performance scores as output. This method is useful when the Pareto points are sparse and the front assumes a convex shape if better performance corresponds to lower measure values (e.g. classification error), or a concave shape otherwise (e.g. classification accuracy). The estimated Pareto front will include points for a number of features ranging from 1 up to the maximum number found in the empirical Pareto front.

Returns

A data.table::data.table with columns the number of features and the performance that together form the Pareto front.

---

Method knee_points()

This function implements various knee point identification (KPI) methods, which select points in the Pareto front, such that an optimal trade-off between performance and number of features is achieved. In most cases, only one such point is returned.

Usage

EnsembleFSResult$knee_points(method = "NBI", type = "empirical")

Arguments

method

(character(1))
Type of method to use to identify the knee point. See details.

type

(character(1))
Specifies the type of Pareto front to use for the identification of the knee point. See pareto_front() method for more details.

Details

The available KPI methods are:

• "NBI" (default): The Normal-Boundary Intersection method is a geometry-based method which calculates the perpendicular distance of each point from the line connecting the first and last points of the Pareto front. The knee point is determined as the Pareto point with the maximum distance from this line, see Das (1999).

Returns

A data.table::data.table with the knee point(s) of the Pareto front.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

EnsembleFSResult$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

References

Das, I (1999). “On characterizing the 'knee' of the Pareto curve based on normal-boundary intersection.” Structural Optimization, 18(1-2), 107–115. ISSN 09344373.

Examples

  efsr = ensemble_fselect(
    fselector = fs("rfe", n_features = 2, feature_fraction = 0.8),
    task = tsk("sonar"),
    learners = lrns(c("classif.rpart", "classif.featureless")),
    init_resampling = rsmp("subsampling", repeats = 2),
    inner_resampling = rsmp("cv", folds = 3),
    measure = msr("classif.ce"),
    terminator = trm("none")
  )

  # contains the benchmark result
  efsr$benchmark_result

  # contains the selected features for each iteration
  efsr$result

  # returns the stability of the selected features
  efsr$stability(stability_measure = "jaccard")

  # returns a ranking of all features
  head(efsr$feature_ranking())

  # returns the empirical pareto front (nfeatures vs error)
  efsr$pareto_front()

mlr3fselect documentation built on Oct. 30, 2024, 9:19 a.m.