This is the result container object returned by
A BenchmarkResult consists of the data row-binded data of multiple ResampleResults,
which can easily be re-constructed.
Note that all stored objects are accessed by reference. Do not modify any object without cloning it first.
Table with data for one resampling iteration per row: Task, Learner, Resampling, iteration (
integer(1)), Prediction, and the unique
character(1)) of the corresponding ResampleResult.
Additional columns are kept in the resulting object.
Internal data storage with one row per resampling iteration. Can be joined with
$rr_data by joining on column
We discourage users to directly work with this table.
Package develops on the other hand may opt to add additional columns here. These columns are preserved in all mutators.
Internal data storage with one row per ResampleResult. Can be joined with
$data by joining on column
Not used in
mlr3 directly, but can be exploited by add-on packages.
Package develops may opt to add additional columns here. These columns are preserved in all mutators.
Task type of objects in the
All stored objects (Task, Learner, Prediction) in a single
BenchmarkResult are required to have the same task type, e.g.,
Table of used tasks with three columns:
Table of used learners with three columns:
Table of used resamplings with three columns:
Returns the number of stored ResampleResults.
Vector of unique hashes of all included ResampleResults.
aggregate(measures = NULL, ids = TRUE, uhashes = FALSE, params = FALSE, conditions = FALSE)
(list of Measure,
Returns a result table where resampling iterations are combined into ResampleResults. A column with the aggregated performance score is added for each Measure, named with the id of the respective measure.
For convenience, the following parameters can be set to extract more information from the returned ResampleResult:
Adds the uhash values of the ResampleResult as extra character column
Adds object ids (
"resampling_id") as extra character columns.
Adds the hyperparameter values as extra list column
You can unnest them with
Adds the number of resampling iterations with at least one warning as extra integer column
the number of resampling iterations with errors as extra integer column
score(measures = NULL, ids = TRUE)
(list of Measure,
Returns a table with one row for each resampling iteration, including all involved objects: Task, Learner, Resampling, iteration number (
integer(1)), and Prediction.
ids is set to
TRUE, character column of extracted ids are added to the table for convenient filtering:
Additionally calculates the provided performance measures and binds the performance as extra columns.
These columns are named using the id of the respective Measure.
resample_result(i = NULL, uhash = NULL)
character(1)) -> ResampleResult
Retrieve the i-th ResampleResult, by position or by unique hash
uhash are mutually exclusive.
Fuses a second BenchmarkResult into itself, mutating the BenchmarkResult in-place. If
NULL, simply returns
Returns a copy of the internal data.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
set.seed(123) learners = list( lrn("classif.featureless", predict_type = "prob"), lrn("classif.rpart", predict_type = "prob") ) design = benchmark_grid( tasks = list(tsk("sonar"), tsk("spam")), learners = learners, resamplings = rsmp("cv", folds = 3) ) print(design) bmr = benchmark(design) print(bmr) bmr$tasks bmr$learners # first 5 individual resamplings head(as.data.table(bmr, measures = c("classif.acc", "classif.auc")), 5) # aggregate results bmr$aggregate() # aggregate results with hyperparameters as separate columns mlr3misc::unnest(bmr$aggregate(params = TRUE), "params") # extract resample result for classif.rpart rr = bmr$aggregate()[learner_id == "classif.rpart", resample_result][] print(rr) # access the confusion matrix of the first resampling iteration rr$predictions()[]$confusion
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.