nrcv_rusranger: Nested Cross Validation for Hyperparameter Search

In ampel-leipzig/rusranger: Modified ranger implementation to support random-under-sampling

Nested Cross Validation for Hyperparameter Search

Description

Run a grid search in a nested cross validation.

Usage

nrcv_rusranger(
  x,
  y,
  searchspace,
  nouterfolds = 5,
  ninnerfolds = 5,
  nrepcv = 2,
  ...
)

Arguments

x	matrix/data.frame, feature matrix, see ranger() for details.
y	numeric/factor, classification labels, see ranger() for details.
searchspace	data.frame, hyperparameters to tune. Column names have to match the argument names of ranger()/rusranger().
nouterfolds	integer(1), number of outer cross validation folds.
ninnerfolds	integer(1), number of inner cross validation folds.
nrepcv	integer(1), number repeats of inner cross validations.
...	further arguments passed to gs_rusranger().

Value

list, with an element per nouterfolds containing the following subelements:

• model selected ranger model.

• indextrain index of the used training items.

• indextest index of the used test items.

• prediction predictions results.

• truth original labels/classes.

• performance resulting performance (AUC).

• selectedparams select hyperparameters.

• gridsearch data.frame, results of the grid search.

• nouterfolds integer(1).

• ninnerfolds integer(1).

• nrepcv integer(1).

Note

The reported performance could slightly differ from the median performance in the reported gridsearch. After the gridsearch rusranger is trained again with the best hyperparameters which results in a new subsampling.

Examples

set.seed(20220324)
iris <- subset(iris, Species != "setosa")
searchspace <- expand.grid(
   mtry = c(2, 3),
   num.trees = c(500, 1000)
)
## n(outer|inner) folds and nrepcv are too low for real world applications,
## and are just used for demonstration and to keep the run time of the examples
## low
nrcv_rusranger(
    iris[-5], as.numeric(iris$Species == "versicolor"),
    searchspace = searchspace, nouterfolds = 3, ninnerfolds = 3, nrepcv = 1
)

ampel-leipzig/rusranger documentation built on April 28, 2022, 3:22 a.m.