R/doc_error_handling.R

#' @title Error handling for mlrMBO
#'
#' @description
#' There are multiple types of errors that can occur during one optimization
#' process. mlrMBO tries to handle most of them as smart as possible.
#'
#' The target function could
#' \itemize{
#'   \item{1}{The target function returns NA(s) or NaN(s) (plural for the multi-objective case).}
#'   \item{2}{The target function stops with an error.}
#'   \item{3}{The target function does not return at all (infinite or very long execution time).}
#'   \item{4}{The target function crashes the whole R process.}
#'   \item{5}{The surrogate machine learning model might crash.
#'     Kriging quite often can run into numerical problems.}
#'   \item{6}{The proposal mechanism - in multi-point or single point mode - produces
#'     a point which is either close to another candidate point in the same iteration or
#'     an already visited point in a previous iteration.}
#'   \item{7}{The mbo process exits / stops / crashes itself. Maybe because it hit a walltime.}
#' }
#'
#'
#' \strong{Mechanism I - Objective value imputation}
#' Issues 1-4 all have in common that the optimizer does not obtain a useful
#' objective value. 3-4 are problematic, because we completely lose control of the R process.
#' We are currently only able to handle them, if you are parallelizing your optimization
#' via \code{\link[parallelMap]{parallelMap}} and use the BatchJobs mode.
#' In this case, you can specify a walltime (handles 3) and the function evaluation is performed
#' in a separate R process (handles 4). A later path might be to allow function evaluation in
#' a separate process in general, with a capping time. If you really need this now, you can always
#' do this yourself.
#'
#' Now back to the problem of invalid objective values. By default, the mbo function stops with an error
#' (if it still has control of the process). But in many cases you still want the algorithm to continue.
#' Hence, mbo allows imputation of bad values via the control option \code{impute.y.fun}.
#'
#' \strong{Logging:} All error messages are logged into the optimization path \code{opt.path} if problems occur.
#'
#'
#' \strong{Mechanism II - The mlr's on.learner.error}
#' If your surrogate learner crashes you can set \code{on.surrogate.error} in \code{\link{makeMBOControl}} to \dQuote{quiet} or \dQuote{warn}.
#' This will set mlr's \code{on.learner.error} for the surrogate.
#' It prevents MBO from crashing in total (issue 5), if the surrogate learner produces an error.
#' As a resort a FailureModel will be returned instead of a the surrogate.
#' Subsequently a random point (or multiple ones) are proposed now for the current iteration.
#' And we pray that we can fit the model again in the next iteration.
#
#' \strong{Logging:} The entry \dQuote{model.error} is set in the \code{opt.path}.
#'
#'
#' \strong{Mechanism III - Filtering of proposed point which are too close}
#'
#' Issue 6 is solved by filtering points that are to close to other proposed points or points already
#' proposed in preceding iterations. Filtering in this context means replacing the proposed points by
#' a randomly generated new point. The heuristics mechanism is (de)activated via the logical
#' \code{filter.proposed.points.tol} parameter of the \code{\link{setMBOControlInfill}} function, which defaults to
#' \code{TRUE}.(closeness of two points is determined via the \code{filter.proposed.points.tol} parameter).
#'
#' \strong{Logging:} The logical entry \dQuote{filtered.point} is set in the opt.path indicating whether the
#' corresponding point was filtered.
#'
#'
#' \strong{Mechanism IV - Continue optimization process}
#'
#' The mechanism is a save-state-then-continue-mechanism, that allows you to continue
#' your optimization after your system or the optimization process crashed for
#' some reason (issue 7). The \code{\link{mbo}} function has the option to save the
#' current state after certain iterations of the main loop on disk via the control
#' option \code{save.on.disk.at} of \code{\link{makeMBOControl}}.
#' Note that this saving mechanism is \emph{disabled} by default.
#' Here you can specify, after which iteration you want the current state to be
#' saved (option \code{save.on.disk.at}). Notice that 0 denotes saving the initial
#' design and \code{iters} + 1 denotes saving the final results.
#' With \code{\link{mboContinue}} you can continue the optimization from the last
#' saved state. This function only requires the path of the saved state.
#'
#' You will get a warning if you turn on saving in general, but not for the the final result, as
#' this seems a bit stupid. \code{save.file.path} defines the path of the RData file where
#' the state is stored. It is overwritten (= extended) in each saving iteration.
#' @name error_handling
NULL
berndbischl/mlrMBO documentation built on Oct. 11, 2022, 1:44 p.m.