ArchiveBatchTuning: Class for Logging Evaluated Hyperparameter Configurations
In mlr-org/mlr3tuning: Hyperparameter Optimization for 'mlr3'
ArchiveBatchTuning R Documentation
Class for Logging Evaluated Hyperparameter Configurations
Description
The ArchiveBatchTuning stores all evaluated hyperparameter configurations and performance scores in a data.table::data.table().
Details
The ArchiveBatchTuning is a container around a data.table::data.table().
Each row corresponds to a single evaluation of a hyperparameter configuration.
See the section on Data Structure for more information.
The archive stores additionally a mlr3::BenchmarkResult ($benchmark_result) that records the resampling experiments.
Each experiment corresponds to to a single evaluation of a hyperparameter configuration.
The table ($data) and the benchmark result ($benchmark_result) are linked by the uhash column.
If the archive is passed to as.data.table(), both are joined automatically.
Data Structure
The table ($data) has the following columns:
One column for each hyperparameter of the search space ($search_space).
One (list-)column for the internal_tuned_values
One column for each performance measure ($codomain).
-
x_domain (list())
Lists of (transformed) hyperparameter values that are passed to the learner.
-
runtime_learners (numeric(1))
Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation.
This does not include potential overhead time.
-
timestamp (POSIXct)
Time stamp when the evaluation was logged into the archive.
-
batch_nr (integer(1))
Hyperparameters are evaluated in batches.
Each batch has a unique batch number.
-
uhash (character(1))
Connects each hyperparameter configuration to the resampling experiment stored in the mlr3::BenchmarkResult.
Analysis
For analyzing the tuning results, it is recommended to pass the ArchiveBatchTuning to as.data.table().
The returned data table is joined with the benchmark result which adds the mlr3::ResampleResult for each hyperparameter evaluation.
The archive provides various getters (e.g. $learners()) to ease the access.
All getters extract by position (i) or unique hash (uhash).
For a complete list of all getters see the methods section.
The benchmark result ($benchmark_result) allows to score the hyperparameter configurations again on a different measure.
Alternatively, measures can be supplied to as.data.table().
The mlr3viz package provides visualizations for tuning results.
S3 Methods
-
as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL)
Returns a tabular view of all evaluated hyperparameter configurations.
ArchiveBatchTuning -> data.table::data.table()
-
x (ArchiveBatchTuning)
-
unnest (character())
Transforms list columns to separate columns. Set to NULL if no column should be unnested.
-
exclude_columns (character())
Exclude columns from table. Set to NULL if no column should be excluded.
-
measures (List of mlr3::Measure)
Score hyperparameter configurations on additional measures.
Super classes
bbotk::Archive -> bbotk::ArchiveBatch -> ArchiveBatchTuning
Public fields
benchmark_result(mlr3::BenchmarkResult)
Benchmark result.
Active bindings
internal_search_space(paradox::ParamSet)
The search space containing those parameters that are internally optimized by the mlr3::Learner.
Methods
Public methods
-
-
-
-
-
-
-
-
Inherited methods
Method new()
Creates a new instance of this R6 class.
Usage
ArchiveBatchTuning$new(
search_space,
codomain,
check_values = FALSE,
internal_search_space = NULL
)
Arguments
search_space(paradox::ParamSet)
Hyperparameter search space. If NULL (default), the search space is
constructed from the paradox::TuneToken of the learner's parameter set
(learner$param_set).
codomain(bbotk::Codomain)
Specifies codomain of objective function i.e. a set of performance measures.
Internally created from provided mlr3::Measures.
check_values(logical(1))
If TRUE (default), hyperparameter configurations are check for validity.
internal_search_space(paradox::ParamSet or NULL)
The internal search space.
Method learner()
Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Learner does not contain a model. Use $learners() to get learners with models.
Usage
ArchiveBatchTuning$learner(i = NULL, uhash = NULL)
Arguments
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
Method learners()
Retrieve list of trained mlr3::Learner objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$learners(i = NULL, uhash = NULL)
Arguments
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
Method learner_param_vals()
Retrieve param values of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$learner_param_vals(i = NULL, uhash = NULL)
Arguments
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
Method predictions()
Retrieve list of mlr3::Prediction objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$predictions(i = NULL, uhash = NULL)
Arguments
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
Method resample_result()
Retrieve mlr3::ResampleResult of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$resample_result(i = NULL, uhash = NULL)
Arguments
i(integer(1))
The iteration value to filter for.
uhash(logical(1))
The uhash value to filter for.
Method print()
Printer.
Usage
ArchiveBatchTuning$print()
Arguments
...(ignored).
Method clone()
The objects of this class are cloneable with this method.
Usage
ArchiveBatchTuning$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
mlr-org/mlr3tuning documentation built on April 14, 2025, 1 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| ArchiveBatchTuning | R Documentation |
Class for Logging Evaluated Hyperparameter Configurations
Description
The ArchiveBatchTuning stores all evaluated hyperparameter configurations and performance scores in a data.table::data.table().
Details
The ArchiveBatchTuning is a container around a data.table::data.table().
Each row corresponds to a single evaluation of a hyperparameter configuration.
See the section on Data Structure for more information.
The archive stores additionally a mlr3::BenchmarkResult ($benchmark_result) that records the resampling experiments.
Each experiment corresponds to to a single evaluation of a hyperparameter configuration.
The table ($data) and the benchmark result ($benchmark_result) are linked by the uhash column.
If the archive is passed to as.data.table(), both are joined automatically.
Data Structure
The table ($data) has the following columns:
One column for each hyperparameter of the search space (
$search_space).One (list-)column for the
internal_tuned_valuesOne column for each performance measure (
$codomain).-
x_domain(list())
Lists of (transformed) hyperparameter values that are passed to the learner. -
runtime_learners(numeric(1))
Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation. This does not include potential overhead time. -
timestamp(POSIXct)
Time stamp when the evaluation was logged into the archive. -
batch_nr(integer(1))
Hyperparameters are evaluated in batches. Each batch has a unique batch number. -
uhash(character(1))
Connects each hyperparameter configuration to the resampling experiment stored in the mlr3::BenchmarkResult.
Analysis
For analyzing the tuning results, it is recommended to pass the ArchiveBatchTuning to as.data.table().
The returned data table is joined with the benchmark result which adds the mlr3::ResampleResult for each hyperparameter evaluation.
The archive provides various getters (e.g. $learners()) to ease the access.
All getters extract by position (i) or unique hash (uhash).
For a complete list of all getters see the methods section.
The benchmark result ($benchmark_result) allows to score the hyperparameter configurations again on a different measure.
Alternatively, measures can be supplied to as.data.table().
The mlr3viz package provides visualizations for tuning results.
S3 Methods
-
as.data.table.ArchiveTuning(x, unnest = "x_domain", exclude_columns = "uhash", measures = NULL)
Returns a tabular view of all evaluated hyperparameter configurations.
ArchiveBatchTuning ->data.table::data.table()
-
x(ArchiveBatchTuning) -
unnest(character())
Transforms list columns to separate columns. Set toNULLif no column should be unnested. -
exclude_columns(character())
Exclude columns from table. Set toNULLif no column should be excluded. -
measures(List of mlr3::Measure)
Score hyperparameter configurations on additional measures.
-
Super classes
bbotk::Archive -> bbotk::ArchiveBatch -> ArchiveBatchTuning
Public fields
benchmark_result(mlr3::BenchmarkResult)
Benchmark result.
Active bindings
internal_search_space(paradox::ParamSet)
The search space containing those parameters that are internally optimized by themlr3::Learner.
Methods
Public methods
Inherited methods
Method new()
Creates a new instance of this R6 class.
Usage
ArchiveBatchTuning$new( search_space, codomain, check_values = FALSE, internal_search_space = NULL )
Arguments
search_space(paradox::ParamSet)
Hyperparameter search space. IfNULL(default), the search space is constructed from the paradox::TuneToken of the learner's parameter set (learner$param_set).codomain(bbotk::Codomain)
Specifies codomain of objective function i.e. a set of performance measures. Internally created from provided mlr3::Measures.check_values(
logical(1))
IfTRUE(default), hyperparameter configurations are check for validity.internal_search_space(paradox::ParamSet or
NULL)
The internal search space.
Method learner()
Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Learner does not contain a model. Use $learners() to get learners with models.
Usage
ArchiveBatchTuning$learner(i = NULL, uhash = NULL)
Arguments
i(
integer(1))
The iteration value to filter for.uhash(
logical(1))
Theuhashvalue to filter for.
Method learners()
Retrieve list of trained mlr3::Learner objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$learners(i = NULL, uhash = NULL)
Arguments
i(
integer(1))
The iteration value to filter for.uhash(
logical(1))
Theuhashvalue to filter for.
Method learner_param_vals()
Retrieve param values of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$learner_param_vals(i = NULL, uhash = NULL)
Arguments
i(
integer(1))
The iteration value to filter for.uhash(
logical(1))
Theuhashvalue to filter for.
Method predictions()
Retrieve list of mlr3::Prediction objects of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$predictions(i = NULL, uhash = NULL)
Arguments
i(
integer(1))
The iteration value to filter for.uhash(
logical(1))
Theuhashvalue to filter for.
Method resample_result()
Retrieve mlr3::ResampleResult of the i-th evaluation, by position or by unique hash uhash.
i and uhash are mutually exclusive.
Usage
ArchiveBatchTuning$resample_result(i = NULL, uhash = NULL)
Arguments
i(
integer(1))
The iteration value to filter for.uhash(
logical(1))
Theuhashvalue to filter for.
Method print()
Printer.
Usage
ArchiveBatchTuning$print()
Arguments
...(ignored).
Method clone()
The objects of this class are cloneable with this method.
Usage
ArchiveBatchTuning$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.