summary.Learner: Summarizing mlr3 Learners
In mlr3summary: Model and Learner Summaries for 'mlr3'
summary.Learner R Documentation
Summarizing mlr3 Learners
Description
summary method for mlr3::Learner.
The output can be tailored via the control argument, see summary_control.
Usage
## S3 method for class 'Learner'
summary(object, resample_result = NULL, control = summary_control(), ...)
## S3 method for class 'GraphLearner'
summary(object, resample_result = NULL, control = summary_control(), ...)
## S3 method for class 'summary.Learner'
print(x, digits = NULL, n_important = NULL, hide = NULL, ...)
Arguments
object
(mlr3::Learner)
trained model of class Learner.
resample_result
(mlr3::ResampleResult)
outcome of resample. If NULL (default), no residuals, performances, etc.
are derived.
control
(summary_control)
a list with control parameters, see summary_control.
...
(any)
further arguments passed to or from other methods.
x
(summary.Learner)
an object of class "summary.Learner", usually a result of a call to summary.Learner.
digits
(numeric(1))
the number of digits to use when printing.
n_important
(numeric(1))
number of important variables to be displayed.
If NULL, x$control$n_important is used.
hide
(character)
Names of paragraphs which should not be part of the summary.
Possible values are "general", "residuals", "performance", "complexity", "fairness",
"importance", "effect". If NULL, no paragraph is hided.
Details
This function can be parallelized with the future package.
One job is one resampling iteration, and all jobs are sent to an apply function
from future.apply in a single batch.
To select a parallel backend, use future::plan().
Value
summary.Learner returns an object of class "summary.Learner", a list with the following entries.
task_type: The type of task, either classif (classification) or regr (regression).
target_name: The name of the target variable.
feature_names: The names of the features.
classes: The classes of the target variable. NULL if regression task.
resample_info: Information on the resample objects, strategy type and hyperparameters.
residuals: Vector of hold-out residuals over the resampling iterations of resample_result.
For regression models, residuals are the difference between true and predicted outcome.
For classifiers with probabilities, the residuals are the difference
between predicted probabilities and a one-hot-encoding of the true class.
For hard-label classifier, a confusion_matrix is shown instead of residuals.
confusion_matrix: Confusion matrix of predicted vs. true classes.
Alternative to residuals, in case of hard-label classification.
performance: Vector of aggregated performance measures over the iterations of resample_result.
The arrows display whether lower or higher values are better.
(micro/macro) displays whether it is a micro or macro measure.
For macro aggregation, measures are computed
for each iteration separately before averaging.
For micro aggregation, measures are computed across all iterations.
See Bischl et al. (2024), for details.
performance_sd: Vector of standard deviations of performance measures
over the iterations of resample_result.
The arrows display whether lower or higher values are better.
(micro/macro) displays whether it is a micro or macro measure.
fairness: Vector of aggregated fairness measures over the iterations of resample_result.
The arrows display whether lower or higher values are better.
(micro/macro) displays whether it is a micro or macro measure.
fairness_sd: Vector of standard deviations of fairness measures
over the iterations of resample_result.
The arrows display whether lower or higher values are better.
(micro/macro) displays whether it is a micro or macro measure (see details above).
importances: List of data.table that display the feature importances
per importance measure. Given are the means and standard deviations
over the resampling iterations of resample_result.
Higher average values display higher importance of a feature.
effects: List of data.tables that display the feature effects
per effect method. Given are the mean effects
over the resampling iterations of resample_result for a maximum of
5 grid points. For binary classifiers, effects are only displayed for
the positively-labeled class.
For multi-class, effect plots are displayed separately for each class.
For categorical features, the factor levels of the feature determine
the ordering of the bars.
complexity: List of vectors that display the complexity values
per complexity measure for each resampling iteration.
control: summary_control used as an input for summary.Learner.
For details on the performance measures, complexity measures, feature
importance and feature effect methods, see summary_control.
References
Bischl, Bernd, Sonabend, Raphael, Kotthoff, Lars, Lang, Michel (2024).
Applied machine learning using mlr3 in R.
Chapman and Hall/CRC.
ISBN 9781003402848, https://mlr3book.mlr-org.com/.
Examples
if (require("mlr3")) {
tsk_iris = tsk("iris")
lrn_rpart = lrn("classif.rpart", predict_type = "prob")
lrn_rpart$train(task = tsk_iris)
rsmp_cv3 = rsmp("cv", folds = 3L)
rr = resample(tsk_iris, lrn_rpart, rsmp_cv3, store_model = TRUE)
summary(lrn_rpart, rr)
}
mlr3summary documentation built on May 29, 2024, 2:44 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| summary.Learner | R Documentation |
Summarizing mlr3 Learners
Description
summary method for mlr3::Learner.
The output can be tailored via the control argument, see summary_control.
Usage
## S3 method for class 'Learner'
summary(object, resample_result = NULL, control = summary_control(), ...)
## S3 method for class 'GraphLearner'
summary(object, resample_result = NULL, control = summary_control(), ...)
## S3 method for class 'summary.Learner'
print(x, digits = NULL, n_important = NULL, hide = NULL, ...)
Arguments
object |
(mlr3::Learner) |
resample_result |
(mlr3::ResampleResult) |
control |
( |
... |
(any) |
x |
( |
digits |
(numeric(1)) |
n_important |
(numeric(1)) |
hide |
(character) |
Details
This function can be parallelized with the future package.
One job is one resampling iteration, and all jobs are sent to an apply function
from future.apply in a single batch.
To select a parallel backend, use future::plan().
Value
summary.Learner returns an object of class "summary.Learner", a list with the following entries.
task_type: The type of task, either
classif(classification) orregr(regression).target_name: The name of the target variable.
feature_names: The names of the features.
classes: The classes of the target variable. NULL if regression task.
resample_info: Information on the resample objects, strategy type and hyperparameters.
residuals: Vector of hold-out residuals over the resampling iterations of
resample_result. For regression models, residuals are the difference between true and predicted outcome. For classifiers with probabilities, the residuals are the difference between predicted probabilities and a one-hot-encoding of the true class. For hard-label classifier, aconfusion_matrixis shown instead ofresiduals.confusion_matrix: Confusion matrix of predicted vs. true classes. Alternative to
residuals, in case of hard-label classification.performance: Vector of aggregated performance measures over the iterations of
resample_result. The arrows display whether lower or higher values are better. (micro/macro) displays whether it is a micro or macro measure. For macro aggregation, measures are computed for each iteration separately before averaging. For micro aggregation, measures are computed across all iterations. See Bischl et al. (2024), for details.performance_sd: Vector of standard deviations of performance measures over the iterations of
resample_result. The arrows display whether lower or higher values are better. (micro/macro) displays whether it is a micro or macro measure.fairness: Vector of aggregated fairness measures over the iterations of
resample_result. The arrows display whether lower or higher values are better. (micro/macro) displays whether it is a micro or macro measure.fairness_sd: Vector of standard deviations of fairness measures over the iterations of
resample_result. The arrows display whether lower or higher values are better. (micro/macro) displays whether it is a micro or macro measure (see details above).importances: List of
data.tablethat display the feature importances per importance measure. Given are the means and standard deviations over the resampling iterations ofresample_result. Higher average values display higher importance of a feature.effects: List of
data.tables that display the feature effects per effect method. Given are the mean effects over the resampling iterations ofresample_resultfor a maximum of 5 grid points. For binary classifiers, effects are only displayed for the positively-labeled class. For multi-class, effect plots are displayed separately for each class. For categorical features, the factor levels of the feature determine the ordering of the bars.complexity: List of vectors that display the complexity values per complexity measure for each resampling iteration.
control: summary_control used as an input for
summary.Learner.
For details on the performance measures, complexity measures, feature importance and feature effect methods, see summary_control.
References
Bischl, Bernd, Sonabend, Raphael, Kotthoff, Lars, Lang, Michel (2024). Applied machine learning using mlr3 in R. Chapman and Hall/CRC. ISBN 9781003402848, https://mlr3book.mlr-org.com/.
Examples
if (require("mlr3")) {
tsk_iris = tsk("iris")
lrn_rpart = lrn("classif.rpart", predict_type = "prob")
lrn_rpart$train(task = tsk_iris)
rsmp_cv3 = rsmp("cv", folds = 3L)
rr = resample(tsk_iris, lrn_rpart, rsmp_cv3, store_model = TRUE)
summary(lrn_rpart, rr)
}
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.