makeBmrsBatch: make a (multicore) mlr benchmark experiment of your...

In pokyah/agrometeoR: Spatial interpolation for Agromet project @ CRA-W

Description

make a (multicore) mlr benchmark experiment of your learner(s) on your task(s)

Usage

makeBmrsBatch(tasks, learners, measures = list(rmse), keep.pred = TRUE,
  models = FALSE, level = "mlr.benchmark", resamplings = "LOO",
  cpus = 4, temp_dir = NULL, prefix = NULL, groupSize = NULL,
  removeTemp = FALSE, crash = FALSE)

Arguments

tasks	a list which elements are object of class mlr::makeRegrTask()
learners	a list which elements are object of class mlr::makeLearner()
measures	a list of the mlr performance metrics you want to get. Default is mlr::list(rmse)
keep.pred	a boolean specifying if you want to keep the bmr preds. defaut = TRUE. Necessary to further analyze benchamrk performances
models	a boolean specifying if you want to keep the bmr models. defaut = FALSE
level	a character specifying the paralelllization level. Default = "mlr.benchmark"
cpus	an integer specifying the number of cpus to use for the benchamrk. Default is 4
temp_dir	a character specifying the path of an exising directory where you want to save the bmr temporary outpus.
prefix	a character specifying the prefix you want to add to each bmr temporary file name.
groupSize	a numeric specifying the number of tasks you want to benchamrk in a single batch. If NULL the value will be set to the length of tasks argument. Default is NULL
removeTemp	a boolean specifying if the temporary .rds generated by the function must be deleted at the end of the process.
crash	a boolean. TRUE if you want the function crash and stop. FALSE if yo uwant the function to handle the error and continue. This is based on the mlr::configureMlr(on.learner.error = "warn") Default to FALSE
resampling	a character specifying the type of mlr's Cross-Validation strategy. Default = "LOO"

Details

The function handles learners error. See mlr::configureMlr().

Value

A 2 elements named list

• snitch : a boolean. Is TRUE if function has provided the expected result. Is FALSE is function throws an error

• output : a named list which elements are :

  • value : an element of class mlr::benchmark()

  • condition : a character specifying the condition encountered by the function : success, warning, or error.

  • message : a character specifying the message relative to the condition.

Author(s)

Thomas Goossens

Examples

## Not run: 
# load magrittr for pipe use : %>%
library(magrittr)

# create the dataset
myDataset = makeDataset(
  dfrom = "2017-03-04T15:00:00Z",
  dto = "2017-03-04T18:00:00Z",
  sensor = "tsa")

# extract the list of hourly sets of records
myDataset = myDataset$output$value

# create the tasks
myTasks = purrr::map(myDataset, makeTask, target = "tsa")

# extract the used sids of each task from the outputs
myUsedSids = myTasks %>% purrr::modify_depth(1, ~.$output$stations$used)

# extract the tasks from the outputs
myTasks = myTasks %>% purrr::modify_depth(1, ~.$output$value$task)

# Conduct a batch of benchmarks experiments without saving temp files
myBmrsBatch = makeBmrsBatch(
tasks = myTasks,
learners = agrometeorLearners,
measures = list(mlr::rmse),
keep.pred = TRUE,
models = FALSE,
groupSize  = NULL,
level = "mlr.benchmark",
resamplings = "LOO",
cpus = 1,
prefix = NULL,
temp_dir = NULL,
removeTemp = FALSE,
crash = FALSE)

# Keep the relevant information
myBmrsBatch = myBmrsBatch$output$value

# make a plot from the myBmrsBatch
mlr::plotBMRBoxplots(myBmrsBatch,
  measure = mlr::rmse,
  order.lrn = getBMRLearnerIds(myBmrsBatch),
  pretty.names = FALSE)

## End(Not run)

pokyah/agrometeoR documentation built on May 26, 2019, 7 p.m.