plotMboHyperparams: Benchmark and plot 'mlrMBO::mbo()' optimization runs to...

In matthiasgruber/EBO: Easy Benchmarks of Optimization Algorithms

Benchmark and plot mlrMBO::mbo() optimization runs to investigate hyperparameters

Description

This functions benchmarks the mlrMBO::mbo() function with different configurations and then plots them wrt the hyperparameters.

Usage

plotMboHyperparams(
  task,
  funcEvals,
  psParamPlot,
  resolution,
  repls,
  showInfo = TRUE,
  ncpus = NA,
  seed = 1
)

Arguments

funcEvals	[integer(1)] Define the amount of black-box function evaluations.
psParamPlot	[ParamHelpers::ParamSet()] Parameter space of the hyperparameters to investigate.
resolution	[integer(1) | integer(2)] The size of the grid for investigating the hyperparameter effect.
repls	[integer(1)] Define how often each configuration is replicated for the benchmark. Default is ten.
showInfo	[logical(1)] Should information be plotted? Default is 'TRUE'.
ncpus	[numeric(1)] Define how many cpu cores are used for the benchmark. Default is NA, which uses all cores minus one.
seed	[numeric(1)] Define the seed used for the computation. Will be set by batchtools. Which means the jobs get the seed plus the job.id as their unique seed. Default is one.

Value

A 1D or 2D ggplot2 object to investigate the effect of the hyperparameters.

Examples

## Not run: 
set.seed(1)
data <- data.frame(a = runif(50,10,5555), b = runif(50,-30000,-500),
                   c = runif(50,0,1000),
                   d = sample(c("nitrogen","air","argon"), 50, replace = TRUE))
data$ratio <- rowSums(data[,1:3]^2)
data$ratio <- data$ratio/max(data$ratio)
colnames(data) <- c("power", "time", "pressure", "gas","ratio")
psOpt = ParamHelpers::makeParamSet(
  ParamHelpers::makeIntegerParam("power", lower = 10, upper = 5555),
  ParamHelpers::makeIntegerParam("time", lower = -30000, upper = -500),
  ParamHelpers::makeNumericParam("pressure", lower = 0, upper = 1000),
  ParamHelpers::makeDiscreteParam("gas", values = c("nitrogen", "air", "argon"))
)

task = task(
  simulation = "regr.randomForest",
  data = data,
  target = "ratio",
  psOpt = psOpt,
  minimize = FALSE
)

funcEvals = 10

psParamPlot = ParamHelpers::makeParamSet(
  ParamHelpers::makeDiscreteParam("surrogate", values = ("regr.randomForest")),
  ParamHelpers::makeDiscreteParam("crit", values = ("makeMBOInfillCritAdaCB")),
  ParamHelpers::makeIntegerParam("cb.lambda.start", lower = 5, upper = 15,
                                 requires = quote(crit == "makeMBOInfillCritAdaCB")),
  ParamHelpers::makeNumericParam("cb.lambda.end", lower = 1, upper = 5,
                                 requires = quote(crit == "makeMBOInfillCritAdaCB"))
)

resolution = 2

repls = 2

showInfo = TRUE

ncpus = NA

seed = 1

contourPlot = plotMboHyperparams(task, funcEvals, psParamPlot, resolution,
                                 repls, showInfo, ncpus, seed)

## End(Not run)

matthiasgruber/EBO documentation built on May 17, 2022, 3:19 p.m.