error_handling: Error handling for mlrMBO

Description

Description

There are multiple types of errors that can occur during one optimization process. mlrMBO tries to handle most of them as smart as possible.

The target function could

Mechanism I - Objective value imputation Issues 1-4 all have in common that the optimizer does not obtain a useful objective value. 3-4 are problematic, because we completely lose control of the R process. We are currently only able to handle them, if you are parallelizing your optimization via parallelMap and use the BatchJobs mode. In this case, you can specify a walltime (handles 3) and the function evaluation is performed in a separate R process (handles 4). A later path might be to allow function evaluation in a separate process in general, with a capping time. If you really need this now, you can always do this yourself.

Now back to the problem of invalid objective values. By default, the mbo function stops with an error (if it still has control of the process). But in many cases you still want the algorithm to continue. Hence, mbo allows imputation of bad values via the control option impute.y.fun.

Logging: All error messages are logged into the optimization path opt.path if problems occur.

Mechanism II - The mlr's on.learner.error This produces a FailureModel for the surrogate, if it crashed (issue 6). A random point (or multiple ones) are proposed now for the current iteration. And we pray that we can fit the model again in the next iteration. Logging: The entry “model.error” is set in the opt.path.

Mechanism III - Filtering of proposed point which are too close

Issue 6 is solved by filtering points that are to close to other proposed points or points already proposed in preceding iterations. Filtering in this context means replacing the proposed points by a randomly generated new point. The heuristics mechanism is (de)activated via the logical filter.proposed.points.tol parameter of the setMBOControlInfill function, which defaults to TRUE.(closeness of two points is determined via the filter.proposed.points.tol parameter).

Logging: The logical entry “filtered.point” is set in the opt.path indicating whether the corresponding point was filtered.

Mechanism IV - Continue optimization process

The mechanism is a save-state-then-continue-mechanism, that allows you to continue your optimization after your system or the optimization process crashed for some reason (issue 7). The mbo function has the option to save the current state after certain iterations of the main loop on disk via the control option save.on.disk.at of makeMBOControl. Note that this saving mechanism is disabled by default. Here you can specify, after which iteration you want the current state to be saved (option save.on.disk.at). Notice that 0 denotes saving the initial design and iters + 1 denotes saving the final results. With mboContinue you can continue the optimization from the last saved state. This function only requires the path of the saved state.

You will get a warning if you turn on saving in general, but not for the the final result, as this seems a bit stupid. save.file.path defines the path of the RData file where the state is stored. It is overwritten (= extended) in each saving iteration.


mlrMBO documentation built on June 25, 2018, 9:04 a.m.