Description Usage Arguments Details Value Note Author(s) References See Also Examples
Uses a surrogate modeled augmented Lagrangian (AL) system to optimize an objective function (blackbox or known and linear) under unknown multiple (blackbox) constraints via expected improvement (EI) and variations; a comparator based on EI with constraints is also provided
1 2 3 4 5 6 7 8 9 10 11  optim.auglag(fn, B, fhat = FALSE, equal = FALSE, ethresh = 1e2,
slack = FALSE, cknown = NULL, start = 10, end = 100,
Xstart = NULL, sep = TRUE, ab = c(3/2, 8), lambda = 1, rho = NULL,
urate = 10, ncandf = function(t) { t }, dg.start = c(0.1, 1e06),
dlim = sqrt(ncol(B)) * c(1/100, 10), Bscale = 1, ey.tol = 1e2,
N = 1000, plotprog = FALSE, verb = 2, ...)
optim.efi(fn, B, fhat = FALSE, cknown = NULL, start = 10, end = 100,
Xstart = NULL, sep = TRUE, ab = c(3/2,8), urate = 10,
ncandf = function(t) { t }, dg.start = c(0.1, 1e6),
dlim = sqrt(ncol(B))*c(1/100,10), Bscale = 1, plotprog = FALSE,
verb = 2, ...)

fn 
function of an input ( 
B 
2column 
fhat 
a scalar logical indicating if the objective function should
be modeled with a GP surrogate. The default of 
equal 
an optional vector containing zeros and ones, whose length equals the number of
constraints, specifying which should be treated as equality constraints ( 
ethresh 
a threshold used for equality constraints to determine validity for progress measures; ignored if there are no equality constraints 
slack 
A scalar logical indicating if slack variables, and thus exact EI
calculations should be used. The default of 
cknown 
A optional positive integer vector specifying which of the constraint
values returned by 
start 
positive integer giving the number of random starting locations before
sequential design (for optimization) is performed; 
end 
positive integer giving the total number of evaluations/trials in the
optimization; must have 
Xstart 
optional matrix of starting design locations in lieu of, or in addition to,

sep 
The default 
ab 
prior parameters; see 
lambda 

rho 
positive scalar initial quadratic penalty parameter in the augmented Lagrangian; the default setting of 
urate 
positive integer indicating how many optimization trials should pass before each MLE/MAP update is performed for GP correlation lengthscale parameter(s) 
ncandf 
function taking a single integer indicating the optimization trial number 
dg.start 
2vector giving starting values for the lengthscale and nugget parameters of the GP surrogate model(s) for constraints 
dlim 
2vector giving bounds for the lengthscale parameter(s) under MLE/MAP inference,
thereby augmenting the prior specification in 
Bscale 
scalar indicating the relationship between the sum of the inputs, 
ey.tol 
a scalar proportion indicating how many of the EIs
at 
N 
positive scalar integer indicating the number of Monte Carlo samples to be used for calculating EI and EY 
plotprog 

verb 
a nonnegative integer indicating the verbosity level; the larger the value the more that is printed to the screen 
... 
additional arguments passed to 
These subroutines support a suite of methods used to optimize challenging constrained problems from Gramacy, et al. (2016); and from Picheny, et al., (2016) see references below.
Those schemes hybridize Gaussian process based surrogate modeling and expected
improvement (EI; Jones, et., al, 2008) with the additive penalty method (APM)
implemented by the augmented Lagrangian (AL, e.g., Nocedal & Wright, 2006).
The goal is to minimize a (possibly known) linear objective function f(x)
under
multiple, unknown (blackbox) constraint functions satisfying c(x) <= 0
,
which is vectorvalued. The solution here emulates the components of c
with Gaussian process surrogates, and guides optimization by searching the
posterior mean surface, or the EI of, the following composite objective
Y(x) = f(x) + lambda %*% Yc(x) + 1/(2rho) sum(max(0, Yc(x))^2)
where lambda and rho follow updating equations that guarantee convergence to a valid solution minimizing the objective. For more details, see Gramacy, et al. (2016).
A slack variable implementation that allows for exact EI calculations and can accommodate equality constraints, and mixed (equality and inequality) constraints, is also provided. For further details, see Picheny, et al. (2016).
The example below illustrates a variation on the toy problem considered in both papers, which bases sampling on EI. For examples making used of equality constraints, following the Picheny, et al (2016) papers; see the demos listed in the “See Also” section below.
Although it is off by default, these functions allow an unknown objective to
be modeled (fhat = TRUE
), rather than assuming a known linear one. For examples see
demo("ALfhat")
and demo("GSBP")
which illustrate the AL and comparators
in inequality and mixed constraints setups, respectively.
The optim.efi
function is provided as a comparator. This method uses
the same underlying GP models to with the hybrid EI and probability of satisfying
the constraints heuristic from Schonlau, et al., (1998). See demo("GSBP")
and demo("LAH")
for optim.efi
examples and comparisons between
the original AL, the slack variable enhancement(s) on mixed constraint
problems with known and blackbox objectives, respectively
The output is a list
summarizing the progress of the evaluations of the
blackbox under optimization
prog 
vector giving the best valid ( 
obj 
vector giving the value of the objective for the input under consideration at each trial 
X 

C 

d 

df 
if 
lambda 
a 
rho 
a vector of 
This function is under active development, especially the newest features
including separable GP surrogate modeling, surrogate modeling of a
blackbox objective, and the use of slack variables for exact EI calculations and
the support if equality constraints. Also note that, compared with earlier versions, it is now
required to augment your blackbox function (fn
) with an argument named
known.only
. This allows the user to specify if a potentially different
object
(with a subset of the outputs, those that are “known”) gets returned in
certain circumstances. For example, the objective is treated as known in many of our
examples. When a nonnull cknown
object is used, the known.only
flag can be used to return only the outputs which are known.
Older versions of this function provided an argument called nomax
.
The NoMax feature is no longer supported
Robert B. Gramacy [email protected]
Picheny, V., Gramacy, R.B., Wild, S.M., Le Digabel, S. (2016). “Bayesian optimization under mixed constraints with a slackvariable augmented Lagrangian”. Preprint available on arXiv:1605.09466; http://arxiv.org/abs/1605.09466
Gramacy, R.B, Gray, G.A, Lee, H.K.H, Le Digabel, S., Ranjan P., Wells, G., Wild, S.M. (2016) “Modeling an Augmented Lagrangian for Improved Blackbox Constrained Optimization”, Technometrics (with discussion), 58(1), 111. Preprint available on arXiv:1403.4890; http://arxiv.org/abs/1403.4890
Jones, D., Schonlau, M., and Welch, W. J. (1998). “Efficient Global Optimization of Expensive Black Box Functions.” Journal of Global Optimization, 13, 455492.
Schonlau, M., Jones, D.R., and Welch, W. J. (1998). “Global Versus Local Search in Constrained Optimization of Computer Models.” In New Developments and Applications in Experimental Design, vol. 34, 1125. Institute of Mathematical Statistics.
Nocedal, J. and Wright, S.J. (2006). Numerical Optimization. 2nd ed. Springer.
vignette("laGP")
, demo("ALfhat")
for blackbox objective,
demo("GSBP")
for a mixed constraints problem with blackbox objective,
demo("LAH")
for mix constraints with known objective,
optim.step.tgp
for unconstrained optimization;
optim
with method="LBFGSB"
for box constraints, or
optim
with method="SANN"
for simulated annealing
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65  ## this example assumes a known linear objective; further examples
## are in the optim.auglag demo
## a test function returning linear objective evaluations and
## nonlinear constraints
aimprob < function(X, known.only = FALSE)
{
if(is.null(nrow(X))) X < matrix(X, nrow=1)
f < rowSums(X)
if(known.only) return(list(obj=f))
c1 < 1.5X[,1]2*X[,2]0.5*sin(2*pi*(X[,1]^22*X[,2]))
c2 < rowSums(X^2)1.5
return(list(obj=f, c=cbind(c1,c2)))
}
## set bounding rectangle for adaptive sampling
B < matrix(c(rep(0,2),rep(1,2)),ncol=2)
## optimization (primarily) by EI, change 25 to 100 for
## 99% chance of finding the global optimum with value 0.6
library(akima) ## for plotprog=interp
out < optim.auglag(aimprob, B, end=25, plotprog=interp)
## using the slack variable implementation which is a little slower
## but more precise; slack=2 augments random search with LBFGSB
out2 < optim.auglag(aimprob, B, end=25, slack=TRUE)
## Not run:
out3 < optim.auglag(aimprob, B, end=25, slack=2)
## End(Not run)
## for more slack examples and comparison to optim.efi on problems
## involving equality and mixed (equality and inequality) constraints,
## see demo("ALfhat"), demo("GSBP") and demo("LAH")
## for comparison, here is a version that uses simulated annealing
## with the Additive Penalty Method (APM) for constraints
## Not run:
aimprob.apm < function(x, B=matrix(c(rep(0,2),rep(1,2)),ncol=2))
{
## check bounding box
for(i in 1:length(x)) {
if(x[i] < B[i,1]  x[i] > B[i,2]) return(Inf)
}
## evaluate objective and constraints
f < sum(x)
c1 < 1.5x[1]2*x[2]0.5*sin(2*pi*(x[1]^22*x[2]))
c2 < x[1]^2+x[2]^21.5
## return APM composite
return(f + abs(c1) + abs(c2))
}
## use SA; specify control=list(maxit=100), say, to control max
## number of iterations; does not easily facilitate plotting progress
out4 < optim(runif(2), aimprob.apm, method="SANN")
## check the final value, which typically does not satisfy both
## constraints
aimprob(out4$par)
## End(Not run)
## for a version with a modeled objective see demo("ALfhat")

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.