testAddIters: run statistical tests for 'mlrMBO::mbo()' iterations
In matthiasgruber/EBO: Easy Benchmarks of Optimization Algorithms

testAddIters

R Documentation

run statistical tests for `mlrMBO::mbo()` iterations

Description

This function benchmarks the mlrMBO::mbo() function on different iterations for different configuration and run statistical tests for iterations.

Usage

testAddIters(configResults, baseIters = 20, addIters = 10, minimize = TRUE)

Arguments

`baseIters`	[`integer(1)`] Define the number of baseline iterations. Default is 10.
`addIters`	[`integer(1)`] Define the number of additional iterations. Default is 10.
`minimize`	[`logical(1)`] Should the target be minimized? Default is 'TRUE'.

Value

Test results and summaries for effect of iterations.

References

[mlrMBO::mbo()]

Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas and Michel Lang; mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions, Preprint: https://arxiv.org/abs/1703.03373 (2017).

Examples

## Not run: 

set.seed(1)

library(mlrMBO)
library(ParamHelpers)
library(mlr)

#define infillCrit
ctrl = mlrMBO::makeMBOControl()
ctrl = mlrMBO::setMBOControlInfill(ctrl, crit = mlrMBO::makeMBOInfillCritEI())

# define MBO configuration
paramsMBO = data.table::data.table(
 design = list("maximinLHS","randomLHS", "random"),
 amountDesign = list(12),
 control = list(ctrl),
 surrogate = list(mlr::makeLearner("regr.km", predict.type = "se"))
)

namesBoxplot = c("maximinLHS",
                "randomLHS",
                "random")

# define runs of each algorithm
repls = 10

# define function evaluations
funcEvals = 32


data <- data.frame(a = runif(50,10,5555), b = runif(50,-30000,-500),
                   c = runif(50,0,1000))
data$ratio <- rowSums(data[,1:3]^2)
data$ratio <- data$ratio/max(data$ratio)
colnames(data) <- c("power", "time", "pressure","ratio")


psOpt = ParamHelpers::makeParamSet(
  ParamHelpers::makeIntegerParam("power", lower = 10, upper = 5555),
  ParamHelpers::makeIntegerParam("time", lower = -30000, upper = -500),
  ParamHelpers::makeNumericParam("pressure", lower = 0, upper = 1000),
)

task = task(
 simulation = "regr.randomForest",
 data = data,
 target = "ratio",
 psOpt = psOpt,
 minimize = FALSE
)

# generate configData
configResults = generateConfigdata(task, funcEvals = funcEvals, paramsMBO,
                                       namesBoxplot = namesBoxplot, repls = repls)

# analyze effect of additional iterations
testAddIters(configResults, baseIters = 10, addIters = 10, minimize = FALSE)

## End(Not run)

matthiasgruber/EBO documentation built on May 17, 2022, 3:19 p.m.