generateConfigdata: Generate benchmark data of 'mlrMBO::mbo()' optimization runs
In matthiasgruber/EBO: Easy Benchmarks of Optimization Algorithms

generateConfigdata

R Documentation

Generate benchmark data of `mlrMBO::mbo()` optimization runs

Description

This function benchmarks the mlrMBO::mbo() function on different configurations. The resulting data can be used for several EBO functions such as EBO::boxplotCurve(), EBO::testAddIters(), EBO::testConfigs().

Usage

generateConfigdata(
  task,
  funcEvals = 50,
  paramsMBO = NULL,
  namesBoxplot = c("default"),
  repls = 20,
  showInfo = TRUE,
  ncpus = NA,
  seed = 1
)

Arguments

`task`	[`EBO:: task()`] Task defines the problem setting.
`funcEvals`	[`integer(1)`] Define the number of function evaluations. Default is 50.
`paramsMBO`	[`data.table::data.table()`] A data.table containing design, amountDesign, control and surrogate as lists. The data.table has to be defined as the expample below.
`repls`	[`integer(1)`] Define how often each configuration is run for the benchmark. Default is 20
`showInfo`	[`logical(1)`] Should some information be shown in the plot? Default is 'TRUE'.
`ncpus`	[`numeric(1)`] Define how many cpu cores are used for the benchmark. Default is NA, which uses all cores and leave one for netflix.
`seed`	[`numeric(1)`] Define the seed used for the computation. Will be set by `batchtools`. Which means the jobs get the seed plus the job.id as their unique seed. Default is one.
`namesBoxplotCurve`	[`character`] The names for the `mlrMBO` configurations Default is 'default'.
`psOpt`	[`ParamHelpers::ParamSet()`] Collection of parameters and their constraints for optimization.
`simulation`	[`character`] The black box function e.g. model for the `mlrMBO` Default is 'regr.randomForest'.
`minimize`	[`logical(1)`] Should the target be minimized? Default is 'TRUE'.

Value

Benchmark data for each configuration.

References

[mlrMBO::mbo()]

Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas and Michel Lang; mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions, Preprint: https://arxiv.org/abs/1703.03373 (2017).

Examples

## Not run: 

set.seed(1)

library(mlrMBO)
library(ParamHelpers)
library(mlr)

# define infillCrit
ctrl = mlrMBO::makeMBOControl()
ctrl = mlrMBO::setMBOControlInfill(ctrl, crit = mlrMBO::makeMBOInfillCritEI())

# define MBO configuration
paramsMBO = data.table::data.table(
 design = list("maximinLHS","randomLHS", "random"),
 amountDesign = list(12),
 control = list(ctrl),
 surrogate = list(mlr::makeLearner("regr.km", predict.type = "se"))
)

namesBoxplot = c("maximinLHS",
                "randomLHS",
                "random")

# define runs of each algorithm
repls = 10

# define function evaluations
funcEvals = 32


data <- data.frame(a = runif(50,10,5555), b = runif(50,-30000,-500),
                   c = runif(50,0,1000))
data$ratio <- rowSums(data[,1:3]^2)
data$ratio <- data$ratio/max(data$ratio)
colnames(data) <- c("power", "time", "pressure","ratio")


psOpt = ParamHelpers::makeParamSet(
  ParamHelpers::makeIntegerParam("power", lower = 10, upper = 5555),
  ParamHelpers::makeIntegerParam("time", lower = -30000, upper = -500),
  ParamHelpers::makeNumericParam("pressure", lower = 0, upper = 1000),
)

task = task(
 simulation = "regr.randomForest",
 data = data,
 target = "ratio",
 psOpt = psOpt,
 minimize = FALSE
)

# generate configData
configResults = generateConfigdata(task, funcEvals = funcEvals, paramsMBO,
                                       namesBoxplot = namesBoxplot, repls = repls)

## End(Not run)

matthiasgruber/EBO documentation built on May 17, 2022, 3:19 p.m.