This section describes how to xspliner works when some predictors are discrete, or when we deal with classification model.
As before let's explain the approach basing on a random forest model. For this case we use HR_data from breakDown package.
Let's load the data
HR_data <- breakDown::HR_data str(HR_data)
and build random forest in which we predict
average_montly_hours based on
library(randomForest) model_rf <- randomForest(average_montly_hours ~ last_evaluation + salary + satisfaction_level, data = HR_data)
We're going to make some transformation/simplification on
To do so, we need to transform formula passed into
Similarly to continuous variable it is enough to use
xf symbol on
salary variable, i.e. use formula:
average_monthly_hours ~ last_evaluation + xf(salary) + satisfaction_level
We could use the formula for our final GLM model (then we will use basic parameters for xf), but first let's learn what changes it does.
Similarly to continuous variables the first thing we do is to get black box model response on single variable, called effect.
For continuous variable we used PD or ALE plots, which was average model response on predictor value. In a discrete case we use Individual Conditional Expectation (ICE). The construction is simple:
In training data we replace all values for selected predictor with one of the factor levels and then we perform predictions on resulting dataset. The action is repeated for all factor levels. As a result we get $n * m$ predicted values, where $n$ is number of observations in original training dataset and $m$ is number of selected factor levels.
For above example (and
salary variable), we will get
nrow(HR_data) * 3 predicted values, as
salary has 3 levels.
To generate model ICE xspliner uses
pdp::partial(ice = TRUE) function.
To specify additional options for the response, we may customize the effect parameter for
xf, just like it was using xs:
average_monthly_hours ~ last_evaluation + xf(salary, effect = list(...)) + satisfaction_level
Possible parameters are inherited from
pdp::partial function (except
ice that is always TRUE).
How can we use info gathered from above data?
In continuous case, we simplified the effect with spline approximation. The main idea for discrete case is to find out which groups give similar black box model response and merge them into common groups. The final model is built on simpler variables that store some information sourced from black box. As a result the GLM is much easier to interpret (for example we reduce 10-level factor into 3-level one).
How is that implemented in xspliner?
The transformation is based on factorMerger package that "Support Adaptive Post-Hoc Fusing of Groups".
Merging the groups uses just two functions from the package:
In order to customize variable transition, just specified (inherited from above functions) parameters inside
transition parameter of
xf formula symbol. For example to use "fast-adaptive" method for groups merging with optimal partition at GIC statistics value of 4, we set:
xf(salary, transition = list(method = "fast-adaptive", value = 4))
In below example, we will transform
salary predictor with cutting of GIC statistics at value = 2.
As in continuous case we need to use the formula within
library(xspliner) model_xs <- xspline( average_montly_hours ~ last_evaluation + xf(salary, transition = list(value = 2)) + satisfaction_level, model = model_rf ) summary(model_xs)
Checking out the model summary, we can realize that "low" and "medium" values were merged into single level (generating "lowmedium" level).
It can be also found by:
The graphical result if fully sourced from factorMerger. It is enough to run:
xspliner can work with classification problems as well. As the final GLM model can work only with binary classification, the only limit here is the number of levels for predicted value (equals to 2).
Let's check below example based on SVM algorithm (
e1071::svm), and modified iris data.
Preparing data (we drop "setosa" level on Species value):
iris_data <- droplevels(iris[iris$Species != "setosa", ])
library(e1071) library(xspliner) model_svm <- svm(Species ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, data = iris_data, probability = TRUE)
When the base model response variable is of class factor (or integer with two unique values) then xspliner automatically detects classification problem. To force specific model response distribution you can set family and link parameters. In this case we can use xspliner in standard way.
As each predictor is continuous variable, let's transform it with
xs usage on standard parameters, and build the model:
model_xs <- xspline(Species ~ xs(Sepal.Length) + xs(Sepal.Width) + xs(Petal.Length) + xs(Petal.Width), model = model_svm) summary(model_xs)
Simple plot for Petal.Width shows that approximation almost fully covers the PDP.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.