plotBenchmark: Benchmark and plot all optimization algoritms connected to...

In matthiasgruber/EBO: Easy Benchmarks of Optimization Algorithms

Benchmark and plot all optimization algoritms connected to EBO.

Description

This functions benchmarks the optimization algorithms: iRace, spotES, spotDE, spotGE, random, cmaesr and mlrMBO and then plots them as boxplots.

Usage

plotBenchmark(
  task,
  funcEvals = 65,
  paramsMBO = data.table::data.table(NULL),
  paramsCMAESR = data.table::data.table(NULL),
  paramsES = data.table::data.table(NULL),
  paramsDE = data.table::data.table(NULL),
  paramsGE = data.table::data.table(NULL),
  repls = 25,
  showInfo = TRUE,
  ncpus = NA,
  seed = 5
)

Arguments

task	[EBO:: task()] Task defines the problem setting.
funcEvals	[integer(1)] Define the amount of black-box function evaluations.
paramsMBO	[data.table::data.table()] A data.table containing design, amountDesign, control and surrogate as lists. The data.table has to be defined as the expample below.
paramsCMAESR	[data.table::data.table()] A data.table containing the hyperparameters: sigma and lambda. Default is [data.table::data.table(NULL)], which defines the default hyperparameters.
paramsES	[data.table::data.table()] A data.table containing the hyperparameters: nu, mue, sigmaInit, nSigma, mutation, tau, stratReco and objReco. Default is [data.table::data.table(NULL)], which defines the default hyperparameters.
paramsDE	[data.table::data.table()] A data.table containing the hyperparameter: populationSize. Default is [data.table::data.table(NULL)], which defines the default hyperparameters.
paramsGE	[data.table::data.table()] A data.table containing the hyperparameter: populationSize. Default is [data.table::data.table(NULL)], which defines the default hyperparameters.
repls	[integer(1)] Define how often each configuration is replicated for the benchmark. Default is ten.
showInfo	[logical(1)] Should some information be shown in the plot? Default is 'TRUE'.
ncpus	[numeric(1)] Define how many cpu cores are used for the benchmark. Default is NA, which uses all cores minus one.
seed	[numeric(1)] Define the seed used for the computation. Will be set by batchtools. Which means the jobs get the seed plus the job.id as their unique seed. Default is one.

Value

A plot containing one boxplot for each algorithm.

Examples

## Not run: 
  set.seed(1)
data <- data.frame(a = runif(50,10,5555), b = runif(50,-30000,-500),
                  c = runif(50,0,1000),
                  d = sample(c("nitrogen","air","argon"), 50, replace = TRUE),
                  e = sample(c("cat1","cat2","cat3"), 50, replace = TRUE))
data$ratio <- rowSums(data[,1:3]^2)
data$ratio <- data$ratio/max(data$ratio)
colnames(data) <- c("power", "time", "pressure", "gas", "cat","testTarget")
instance = mlr::train(mlr::makeLearner("regr.randomForest"), mlr::makeRegrTask(data = data, target = "testTarget"))

psOpt = ParamHelpers::makeParamSet(
  ParamHelpers::makeIntegerParam("power", lower = 10, upper = 5555),
  ParamHelpers::makeIntegerParam("time", lower = -30000, upper = -500),
  ParamHelpers::makeNumericParam("pressure", lower = 0, upper = 1000),
  ParamHelpers::makeDiscreteParam("gas", values = c("nitrogen", "air", "argon")),
  ParamHelpers::makeDiscreteParam("cat", values = c("cat1","cat2","cat3"))
)

funcEvals = 60

task = task(
  simulation = "regr.randomForest",
  data = data,
  target = "testTarget",
  psOpt = psOpt,
  minimize = FALSE
)
plotBenchmark2 = plotBenchmark(task, funcEvals, repls = 2, seed = 1)

## End(Not run)

matthiasgruber/EBO documentation built on May 17, 2022, 3:19 p.m.