knitr::opts_chunk$set(warning = FALSE) knitr::opts_chunk$set(message = FALSE)
To illustrate application of
auditor we will use dataset "dragons" available in the
DALEX package. The dataset contains characteristics of fictional creatures (dragons), like year of birth, height, weight, etc (see below). The goal is to predict the length of life of dragons (a regression problem).
library(DALEX) data(dragons) head(dragons)
First, we need models to compare. We selected linear regression and random forest because of their different structures. Linear regression model linear relationships between target response and independent variables. While random forest should be able to capture also non-linear relationships between variables.
# Linear regression lm_model <- lm(life_length ~ ., data = dragons) # Random forest library(randomForest) set.seed(59) rf_model <- randomForest(life_length ~ ., data = dragons)
Analysis begins with creation of an explainer object with
explain function from
DALEX package. Explainer wraps a model with its meta-data, such as dataset that was used for training or observed response.
lm_exp <- DALEX::explain(lm_model, label = "lm", data = dragons, y = dragons$life_length) rf_exp <- DALEX::explain(rf_model, label = "rf", data = dragons, y = dragons$life_length)
Next step requires creation of
model_performance objects of each explained model. The function computes chosen performance measures for passed model.
library(auditor) lm_mp <- model_performance(lm_exp) rf_mp <- model_performance(rf_exp) lm_mp
Model performance measures may be plotted together to easily compare model performances. A result further from the center of the plot means a better model performance. Parameter
verbose indicates whether the table with scores should be generated. On the plot scores are inversed and scaled to [0,1].
plot(lm_mp, rf_mp) # alternative: # plot_radar(lm_mp, rf_mp, verbose = FALSE)
There is a possibility to define functions with custom model performance measure.
new_score <- function(object) sum(sqrt(abs(object$residuals))) lm_mp <- model_performance(lm_exp, score = c("mae", "mse", "rec", "rroc"), new_score = new_score) rf_mp <- model_performance(rf_exp, score = c("mae", "mse", "rec", "rroc"), new_score = new_score) plot(lm_mp, rf_mp)
Other methods and plots are described in vignettes:
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.