trainig_test_comparison: Compare performance of model between training and test set

Description Usage Arguments Value Examples

Description

Function training_test_comparison calculates performance of the provided model based on specified measure function. Response of the model is caluclated based on test data, extracted from the explainer and training data, provided by the user. Output can be easily shown with print or plot function.

Usage

1
2
3
4
5
6
7
training_test_comparison(
  champion,
  challengers,
  training_data,
  training_y,
  measure_function = NULL
)

Arguments

champion

- explainer of champion model.

challengers

- explainer of challenger model or list of explainers.

training_data

- data without target column that will be passed to predict function and then to measure function. Keep in mind that they have to differ from data passed to an explainer.

training_y

- target column for training_data

measure_function

- measure function that calculates performance of model based on true observation and prediction. Order of parameters is important and should be (y, y_hat). By default it is RMSE.

Value

An object of the class training_test_comparison.

It is a named list containig:

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
library("mlr")
library("DALEXtra")
task <- mlr::makeRegrTask(
 id = "R",
  data = apartments,
   target = "m2.price"
)
 learner_lm <- mlr::makeLearner(
 "regr.lm"
)
model_lm <- mlr::train(learner_lm, task)
explainer_lm <- explain_mlr(model_lm, apartmentsTest, apartmentsTest$m2.price, label = "LM")

learner_rf <- mlr::makeLearner(
"regr.randomForest"
)
model_rf <- mlr::train(learner_rf, task)
explainer_rf <- explain_mlr(model_rf, apartmentsTest, apartmentsTest$m2.price, label = "RF")

learner_gbm <- mlr::makeLearner(
"regr.gbm"
)
model_gbm <- mlr::train(learner_gbm, task)
explainer_gbm <- explain_mlr(model_gbm, apartmentsTest, apartmentsTest$m2.price, label = "GBM")

data <- training_test_comparison(explainer_lm, list(explainer_gbm, explainer_rf),
                                 training_data = apartments,
                                 training_y = apartments$m2.price)
plot(data)

DALEXtra documentation built on May 9, 2021, 9:07 a.m.