# Classification and discrete predictors In xspliner: Assisted Model Building, using Surrogate Black-Box Models to Train Interpretable Spline Based Additive Models

This section describes how to xspliner works when some predictors are discrete, or when we deal with classification model.

# Qualitative predictors

As before let's explain the approach basing on a random forest model. For this case we use HR_data from breakDown package.

```HR_data <- breakDown::HR_data
str(HR_data)
```

and build random forest in which we predict `average_montly_hours` based on `last_evaluation`, `salary` and `satisfaction_level` predictors.

```library(randomForest)
model_rf <- randomForest(average_montly_hours ~ last_evaluation + salary + satisfaction_level, data = HR_data)
```

We're going to make some transformation/simplification on `salary` variable. To do so, we need to transform formula passed into `xspline` function.

Similarly to continuous variable it is enough to use `xf` symbol on `salary` variable, i.e. use formula:

```average_monthly_hours ~ last_evaluation + xf(salary) + satisfaction_level
```

We could use the formula for our final GLM model (then we will use basic parameters for xf), but first let's learn what changes it does.

## Individual Conditional Expectation for black box model

Similarly to continuous variables the first thing we do is to get black box model response on single variable, called effect.

For continuous variable we used PD or ALE plots, which was average model response on predictor value. In a discrete case we use Individual Conditional Expectation (ICE). The construction is simple:

In training data we replace all values for selected predictor with one of the factor levels and then we perform predictions on resulting dataset. The action is repeated for all factor levels. As a result we get \$n * m\$ predicted values, where \$n\$ is number of observations in original training dataset and \$m\$ is number of selected factor levels.

For above example (and `salary` variable), we will get `nrow(HR_data) * 3` predicted values, as `salary` has 3 levels.

To generate model ICE xspliner uses `pdp::partial(ice = TRUE)` function. To specify additional options for the response, we may customize the effect parameter for `xf`, just like it was using xs:

```average_monthly_hours ~ last_evaluation + xf(salary, effect = list(...)) + satisfaction_level
```

Possible parameters are inherited from `pdp::partial` function (except `ice` that is always TRUE).

How can we use info gathered from above data?

## Merging similar groups with factorMerger

In continuous case, we simplified the effect with spline approximation. The main idea for discrete case is to find out which groups give similar black box model response and merge them into common groups. The final model is built on simpler variables that store some information sourced from black box. As a result the GLM is much easier to interpret (for example we reduce 10-level factor into 3-level one).

How is that implemented in xspliner?

The transformation is based on factorMerger package that "Support Adaptive Post-Hoc Fusing of Groups".

Merging the groups uses just two functions from the package:

• `mergeFactors`
• `getOptimalPartitionDf`

In order to customize variable transition, just specified (inherited from above functions) parameters inside `transition` parameter of `xf` formula symbol. For example to use "fast-adaptive" method for groups merging with optimal partition at GIC statistics value of 4, we set:

```xf(salary, transition = list(method = "fast-adaptive", value = 4))
```

In below example, we will transform `salary` predictor with cutting of GIC statistics at value = 2. As in continuous case we need to use the formula within `xspline` function:

```library(xspliner)
model_xs <- xspline(
average_montly_hours ~ last_evaluation + xf(salary, transition = list(value = 2)) + satisfaction_level,
model = model_rf
)

summary(model_xs)
```

Checking out the model summary, we can realize that "low" and "medium" values were merged into single level (generating "lowmedium" level).

It can be also found by:

```summary(model_xs, "salary")
```

The graphical result if fully sourced from factorMerger. It is enough to run:

```plot_variable_transition(model_xs, "salary")
```

# Quantitative predictors

xspliner can work with classification problems as well. As the final GLM model can work only with binary classification, the only limit here is the number of levels for predicted value (equals to 2).

Let's check below example based on SVM algorithm (`e1071::svm`), and modified iris data.

Preparing data (we drop "setosa" level on Species value):

```iris_data <- droplevels(iris[iris\$Species != "setosa", ])
```

Building SVM:

```library(e1071)
library(xspliner)
model_svm <- svm(Species ~  Sepal.Length + Sepal.Width + Petal.Length + Petal.Width,
data = iris_data, probability = TRUE)
```

When the base model response variable is of class factor (or integer with two unique values) then xspliner automatically detects classification problem. To force specific model response distribution you can set family and link parameters. In this case we can use xspliner in standard way.

As each predictor is continuous variable, let's transform it with `xs` usage on standard parameters, and build the model:

```model_xs <- xspline(Species ~  xs(Sepal.Length) + xs(Sepal.Width) + xs(Petal.Length) + xs(Petal.Width),
model = model_svm)
summary(model_xs)
```

Simple plot for Petal.Width shows that approximation almost fully covers the PDP.

```plot_variable_transition(model_xs, "Petal.Width")
```

## Try the xspliner package in your browser

Any scripts or data that you put into this service are public.

xspliner documentation built on Sept. 26, 2019, 1:02 a.m.