testConfigs: run statistical tests for 'mlrMBO::mbo()' configurations

In matthiasgruber/EBO: Easy Benchmarks of Optimization Algorithms

run statistical tests for mlrMBO::mbo() configurations

Description

This function runs statistical tests for the mlrMBO::mbo() function on different configurations.

Usage

testConfigs(configResults)

Value

Test results and summaries for different mlrMBO configurations.

References

[mlrMBO::mbo()]

Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas and Michel Lang; mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions, Preprint: https://arxiv.org/abs/1703.03373 (2017).

See Also

optimize::plotBenchmark() optimize::plotMboContourPlot()

Examples

## Not run: 

set.seed(1)

library(mlrMBO)
library(ParamHelpers)
library(mlr)

#define infillCrit
ctrl = mlrMBO::makeMBOControl()
ctrl = mlrMBO::setMBOControlInfill(ctrl, crit = mlrMBO::makeMBOInfillCritEI())

# define MBO configuration
paramsMBO = data.table::data.table(
 design = list("maximinLHS","randomLHS", "random"),
 amountDesign = list(12),
 control = list(ctrl),
 surrogate = list(mlr::makeLearner("regr.km", predict.type = "se"))
)

namesBoxplot = c("maximinLHS",
                "randomLHS",
                "random")

# define runs of each algorithm
repls = 10

# define function evaluations
funcEvals = 32


data <- data.frame(a = runif(50,10,5555), b = runif(50,-30000,-500),
                   c = runif(50,0,1000))
data$ratio <- rowSums(data[,1:3]^2)
data$ratio <- data$ratio/max(data$ratio)
colnames(data) <- c("power", "time", "pressure","ratio")


psOpt = ParamHelpers::makeParamSet(
  ParamHelpers::makeIntegerParam("power", lower = 10, upper = 5555),
  ParamHelpers::makeIntegerParam("time", lower = -30000, upper = -500),
  ParamHelpers::makeNumericParam("pressure", lower = 0, upper = 1000),
)

task = task(
 simulation = "regr.randomForest",
 data = data,
 target = "ratio",
 psOpt = psOpt,
 minimize = FALSE
)

# generate configData
configResults = generateConfigdata(task, funcEvals = funcEvals, paramsMBO,
                                       namesBoxplot = namesBoxplot, repls = repls)
# boxplot curve of configs
boxplotCurve(configResults)

# run statistical tests for configs
testConfigs(configResults)

# analyze effect of additional iterations
testAddIters(configResults, baseIters = 10, addIters = 10, minimize = FALSE)

## End(Not run)

matthiasgruber/EBO documentation built on May 17, 2022, 3:19 p.m.