permutationImportance: computes permutation importance

In mmpf: Monte-Carlo Methods for Prediction Functions

Description

computes the change in prediction error from permuting variables.

Usage

permutationImportance(data, vars, y, model, nperm = 100L,
  predict.fun = function(object, newdata) predict(object, newdata =
  newdata), loss.fun = function(x, y) defaultLoss(x, y),
  contrast.fun = function(x, y) x - y)

Arguments

data	a data.frame including both y and vars.
vars	a character vector specifying columns of data to permute.
y	a character vector giving the name of the target/outcome variable.
model	an object with a predict method which returns a vector or matrix. presumably this object represents a model fit.
nperm	positive integer giving the number of times to permute the indicated variables (default is 100).
predict.fun	what function to generate predictions using model. default is the predict method for model. the function must take two arguments, object and newdata and should return a vector or matrix.
loss.fun	what loss function to use to measure prediction errors. default is mean squared-error for ordered predictions and mean misclassification error for unordered prediction errors. this function must take two arguments, “x” and “y”, which operate on the output of predict.fun and data[, y].
contrast.fun	what function to use to contrast the permuted and unpermuted predictions. default is the difference. this function takes two arguments “x” and “y”, which are the output of the loss.fun.

Value

a numeric vector or matrix, depending on contrast.fun and loss.fun, giving the change in prediction error from nperm permutations of vars.

Examples

X = replicate(3, rnorm(100))
y = X %*% runif(3)
data = data.frame(X, y)
fit = lm(y ~ -1 + X1 + X2 + X3, data)

permutationImportance(data, "X1", "y", fit)

Example output

[1] 0.2970126

mmpf documentation built on May 1, 2019, 8:34 p.m.