extract_inner_tuning_archives | R Documentation |
Extract inner tuning archives of nested resampling.
Implemented for mlr3::ResampleResult and mlr3::BenchmarkResult.
The function iterates over the AutoTuner objects and binds the tuning archives to a data.table::data.table()
.
AutoTuner must be initialized with store_tuning_instance = TRUE
and mlr3::resample()
or mlr3::benchmark()
must be called with store_models = TRUE
.
extract_inner_tuning_archives(
x,
unnest = "x_domain",
exclude_columns = "uhash"
)
x |
(mlr3::ResampleResult | mlr3::BenchmarkResult). |
unnest |
( |
exclude_columns |
( |
data.table::data.table()
.
The returned data table has the following columns:
experiment
(integer(1))
Index, giving the according row number in the original benchmark grid.
iteration
(integer(1))
Iteration of the outer resampling.
One column for each hyperparameter of the search spaces.
One column for each performance measure.
runtime_learners
(numeric(1)
)
Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation.
This does not include potential overhead time.
timestamp
(POSIXct
)
Time stamp when the evaluation was logged into the archive.
batch_nr
(integer(1)
)
Hyperparameters are evaluated in batches.
Each batch has a unique batch number.
x_domain
(list()
)
List of transformed hyperparameter values.
By default this column is unnested.
x_domain_*
(any
)
Separate column for each transformed hyperparameter.
resample_result
(mlr3::ResampleResult)
Resample result of the inner resampling.
task_id
(character(1)
).
learner_id
(character(1)
).
resampling_id
(character(1)
).
# Nested Resampling on Palmer Penguins Data Set
learner = lrn("classif.rpart",
cp = to_tune(1e-04, 1e-1, logscale = TRUE))
# create auto tuner
at = auto_tuner(
tuner = tnr("random_search"),
learner = learner,
resampling = rsmp ("holdout"),
measure = msr("classif.ce"),
term_evals = 4)
resampling_outer = rsmp("cv", folds = 2)
rr = resample(tsk("iris"), at, resampling_outer, store_models = TRUE)
# extract inner archives
extract_inner_tuning_archives(rr)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.