evalParamCensus: evaluate hyperparameter config on census data

In SPOTMisc: Misc Extensions for the 'SPOT' Package

evaluate hyperparameter config on census data

Description

evaluate hyperparameter config on census data

Usage

evalParamCensus(
  runNr = "00",
  model = "dl",
  xbest = "xBestOcba",
  k = 30,
  directory = "data",
  target = "age",
  cachedir = "oml.cache",
  task.type = "classif",
  nobs = 10000,
  nfactors = "high",
  nnumericals = "high",
  cardinality = "high",
  prop = 2/3,
  verbosity = 0
)

Arguments

runNr	run number (character)
model	ml/dl model (character)
xbest	best value, e.g., "xBestOcba" or "xbest"
k	number of repeats (integer)
directory	location of the (non-default, e.g., tuned) parameter file
target	"age" or "income_class"
cachedir	cache dir
task.type	task type: "classif" or "regression"
nobs	number of observations
nfactors	factors, e.g., "high"
nnumericals	numericals
cardinality	cardinality
prop	proportion. Default: 2/3
verbosity	verbosity level (0 or 1)

Examples

### These examples require an activated Python environment as described in
### Bartz-Beielstein, T., Rehbach, F., Sen, A., and Zaefferer, M.:
### Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT,
### June 2021. http://arxiv.org/abs/2105.14625.
PYTHON_RETICULATE <- FALSE
if(PYTHON_RETICULATE){
## The following code was used to evaluate the results in the book
## "Hyperparameter Tuning for Machine and Deep Learning with R - A Practical Guide"
## by Bartz, Bartz-Beielstein, Zaefferer, Mersmann:
##
modelList <- list("dl", "cvglmnet",  "kknn", "ranger", "rpart" , "svm", "xgboost")
runNr <- list("100", "Default")
directory <- "../book/data"
for (model in modelList){
  for (run in runNr){    score <- evalParamCensus(model = model,
                               runNr = run,
                               directory = directory,
                               prop=2/3,
                               k=30)
fileName <- paste0(directory, "/", model, run, "Evaluation.RData")
save(score, file = fileName)
 }}
}

SPOTMisc documentation built on Sept. 5, 2022, 5:06 p.m.