mlr_test_helpers: Documentation of mlr3 test helpers

In mlr3: Machine Learning in R - Next Generation

Documentation of mlr3 test helpers

Description

The mlr3 package contains various helper functions to test the validity of objects such as learners. These functions are not contained in the mlr3 namespaces and are instead located in the inst/testthat directory of the source package or the testthat directory of the installed package.

These files can be sourced with the following line of code:

lapply(list.files(system.file("testthat", package = "mlr3"), pattern = "^helper.*\\.[rR]", full.names = TRUE), source)

Other extension packages such as mlr3proba have similar files that can be sourced accordingly.

This manual page documents the most important helper functions that are relevant when users implement their own custom learners.

run_autotest()

This function runs a Learner's automatic test suite.

During the autotests, multiple tasks are generated depending on the properties of the learner. The run_autotest() function then trains the learner on each task and predicts with all supported predict types. (see argument predict_types). To debug, simply run result = run_autotest(learner) and proceed with investigating the task, learner and prediction of the returned result.

For example usages you can look at the autotests in various mlr3 source repositories such as mlr3learners. More information can be found in the inst/testthat/autotest.R file.

Parameters:

• learner (Learner)
  The learner to check.

• N (integer(1))
  The number of rows of the generated tasks.

• exclude (character())
  Each task on which the learner is trained has an id. If for some reason, one or more such tests ought to be disabled, this argument takes in a regular expression that disables all tasks whose id matches the regular expression.

• predict_types (character())
  The predict types of the learner to check. Defaults to all predict typpes of the learner.

• check_replicable (logical(1))
  Whether to check that running the learner twice with the same seed should result in identical predictions. Default is TRUE.

• configure_learner (⁠function(learner, task)⁠)
  Before running a learner on a task, this function allows to change its parameter values depending on the input task.

run_paramtest()

Description:

Checks parameters of mlr3 Learners against parameters defined in the upstream functions of the respective learner. The goal is to detect if parameters have been dropped or added in the upstream implementation. Some learners do not have all of their parameters stored within the learner function that is called during training. Sometimes learners come with a "control" function, e.g. glmnet.control() from package glmnet. Such learners need to be checked as well since they make up the full ParamSet of the respective learner.

To work nicely with the defined ParamSet, certain parameters need to be excluded because these are only present in either the "control" object or the actual top-level function call. Such exclusions should go into argument exclude with a comment for the reason of the exclusion. See examples for more information.

For example usages you can look at the parameter tests in various mlr3 source repositories such as mlr3learners.

Parameters:

• learner (Learner)
  The learner whose parameter set is being checked.

• fun (⁠function()⁠ or list of functions()s)
  The function(s) containing the parameters that must be implemented by the learner.

• exclude (character())
  Argument names that specified through this argument are exempt from checking. This can be used when parameters that are available in the fun function(s) are not implemented in the learner, or when the learner implements additional parameters that are not available in the fun function(s).

• tag (character(1))
  Only parameters that are tagged with this tag are being checked. If NULL (default), all parameters are checked.

expect_learner()

Checks various properties that learners have to satisfy. Used for testing learner implementations, especially if all methods and fields are implement as document.

Parameters

• lrn :: (Learner)
  The learner whose properties are being verified.

• tsk :: (Task)
  Optional argument (default is NULL). If provided, some additional checks are being run that check the compatibility of the learner and task.

• check_man :: (logical(1))
  Whether to check if the learner has a man page.

mlr3 documentation built on Oct. 18, 2024, 5:11 p.m.