forest_rfe: lightweight implementation of RFE using ranger

In talegari/forager: Compute auxiliary information (proximity, dissimilarity, outlyingness, depth) and imputation from tree ensembles on new data

Description

For datasets with large number of predictors, this implementation has these modifications to regular recursive feature elimination procedure:

• Use oob prediction error as a proxy to model performance.

• Build forests ranger on samples of data and average variable importance and oob prediction error.

For a comprehensive RFE procedure with resampling, use rfe

Usage

forest_rfe(dataset, responseVarName, sizes, sampleprop = 0.2,
  nsamples = 10, seed = 1, ...)

Arguments

dataset	(object inheriting data.frame class) A dataframe
responseVarName	(string) Name of the response variable
sizes	(integer vector) Vector of number of variables. When missing, sizes will be sequence of nc/2^i where the sequnce ranges from nc(number of columns) to 2.
sampleprop	(A real number between 0 and 1 or a vector) Proportion of observations. If not a single number and sizes is specified, this vector should have same length as sizes. per sample
nsamples	(positive integer or a vector) Number of samples. If not a single number and sizes is specified, this vector should have same length as sizes.
seed	(positive integer) Seed
...	Arguments to be passed to ranger

Value

A list with:

• (rfeTable) A tibble with three columns:

  • size: Number of variables used

  • ooberror: Out-of-box error of the forest

  • varimp: A list-column where each item is a data.frame with variable names and importance

• (oobchangeTable) A dataframe with five columns sorted by absolute value of the variable 'oepc'.

  • variable: Name of the variable that got removed at some stage

  • size: Number of variables that were considered before removing the variable

  • reducedSize: Number of the variables at next stage. Gives an idea of how many variables were reduced at that stage.

  • oepc: OOB error percentage change

  • importance: Importance of the variable at the stage when the variable was decided to be removed.

References

• RFE using caret

• RFE using scikit learn

Examples

temp <- forest_rfe(iris, "Species")
temp

temp <- forest_rfe(iris
                   , "Species"
                   , sizes = c(4,2)
                   , sampleprop = c(0.2, 0.3)
                   , nsamples = c(20, 30)
                   )
temp

temp <- forest_rfe(iris
                   , "Species"
                   , sizes = c(4,2)
                   , sampleprop = 0.1
                   , nsamples = c(20, 30)
                   )
temp

temp <- forest_rfe(iris
                   , "Species"
                   , sizes = c(4,2)
                   , sampleprop = c(0.2, 0.3)
                   , nsamples = 10
                   )
temp

temp <- forest_rfe(iris
                   , "Species"
                   , sizes = c(4,2)
                   , sampleprop = c(0.2, 0.3)
                   , nsamples = 10
                   , mtry = list(3, 2)
                   , num.trees = list(500, 1000)
                   , case.weights = replicate(2, runif(150), simplify = FALSE)
                   )
temp

talegari/forager documentation built on May 3, 2019, 4:01 p.m.