jackknife: Jackknife test to measure variable importance

In correapvf/caretSDM: Species Distribution Models using caret

Jackknife test to measure variable importance

Description

Create metrics for models with and without each variable.

Usage

jackknife(model, ...)

## S3 method for class 'train'
jackknife(
  model,
  summaryFunction = NULL,
  errorFunction = ci_95,
  param_override_only = NULL,
  param_override_without = NULL,
  progress = FALSE,
  ...
)

## S3 method for class 'jackknife.train'
plot(x, metric = NULL, plot_errorbar = TRUE, ...)

Arguments

model	A model returned by train.
...	ignored
summaryFunction	A Summary function (one of defaultSummary) that calculate the metrics. If NULL, the summaryFunction provided in the model will be used.
errorFunction	A function used to calculate errors across resamples. Default is 95% confidence interval.
param_override_only, param_override_without	A list or data.frame with parameters to override the best tune before they are passed to "train" to create a model with only and without one variable. Useful for models where tuning parameters are dependent of the number of variables (like 'mtry' for randomForest).
progress	logical. Show progress bar?
x	An object from jackknife.
metric	A character, indicating which metric to plot. If NULL, only the first metric is plotted.
plot_errorbar	logical. Should plot error bars?

Details

Multiple models are created using the same method and parameters of the model. Each variable is excluded in turn, and a model created with the remaining variables. Then a model is created using each variable in isolation. Metrics (provided by summaryFunction) are calculated for each model.

Value

A data.table that also inherts "jackknife.train" with metrics for each variable.

Examples

## Not run: 
jackknife.train(model)

# using caret twoClassSummary and standard deviation as errors
jackknife.train(model, summaryFunction = twoClassSummary, errorFunction = sd)

# For models trained with 'rf', override 'mtry' as 1 when training with a single variable
j <- jackknife.train(model, param_override_only = list(mtry=1))
plot(j)

## End(Not run)

correapvf/caretSDM documentation built on June 2, 2022, 8:29 a.m.