summary.xspliner: Summary method for xspliner object

In xspliner: Assisted Model Building, using Surrogate Black-Box Models to Train Interpretable Spline Based Additive Models

Description

Summary method for xspliner object

Usage

## S3 method for class 'xspliner'
summary(object, predictor, ..., model = NULL,
  newdata = NULL, prediction_funs = list(function(object, newdata)
  predict(object, newdata)), env = parent.frame())

Arguments

object	xspliner object
predictor	predictor for xspliner model formula
...	Another arguments passed into model specific method.
model	Original black box model. Providing enables models comparison. See details.
newdata	Data used for models comparison. By default training data used for black box build.
prediction_funs	List of prediction functions for surrogate and black box model. For classification problem, different statistics are displayed based on predictions type. See details section for more info.
env	Environment in which newdata is stored (if not provided as parameter).

Details

The summary output depends strictly on data provided to it.

Standard output for providing only xspliner model (object parameter) return default glm::summary output.

Providing both xspliner model and predictor returns summary details for selecter variable. The following points decribe the rules:

• When variable was quantitative and transformed with fitted spline, the output contain approximation details.

• When variable was qualitative and transformed, factor matching is displayed.

• When variable was not transformed, glm::summary output is displayed for the model.

If both object parameter and model (original black box) was provided, the summary displays comparison of original and surrogate model. The following points decribe the rules (y_s and y_o are predictions of surrogate and original model respectively on provided dataset). When comparing statistic is close to 1, this means surrogate model is similiar to black box one (according to this statistic).

For regression models:

• 1 - Maximum predictions normed-difference

  1 - (max(abs(y_s - y_o)) / diff(range(y_o)))

• R^2 (https://christophm.github.io/interpretable-ml-book/global.html#theory-4)

  1 - sum((y_s - y_o) ^ 2) / sum((y_o - mean(y_o)) ^ 2)

• Mean square errors for each model.

For classification models the result depends on prediction type. When predictions are classified levels:

• Mean predictions similarity

  mean(y_s == y_o)

• Accuracies for each models.

When predictions are response probabilities:

• R^2 as for regression model.

• 1 - Maximum ROC difference

  Calculates maximum of euclidean distances between ROC points for specified thresholds set T. In this imlplementation T is union of breakpoints for each ROC curve.

• 1 - Mean ROC difference Above version using mean instead of max measure.

Examples

library(randomForest)
set.seed(1)
data <- iris
# regression model
iris.rf <- randomForest(Petal.Width ~  Sepal.Length + Petal.Length + Species, data = data)
iris.xs <- xspline(iris.rf)
# Summary of quantitative variable transition
summary(iris.xs, "Sepal.Length")
# Summary of qualitative variable transition
summary(iris.xs, "Species")
# Comparing surrogate with original model (regression)
summary(iris.xs, model = iris.rf, newdata = data)

# Classification model

xspliner documentation built on Sept. 26, 2019, 1:02 a.m.