maxBFGS: BFGS, conjugate gradient, SANN and Nelder-Mead Maximization

In maxLik: Maximum Likelihood Estimation and Related Tools

BFGS, conjugate gradient, SANN and Nelder-Mead Maximization

Description

These functions are wrappers for optim, adding constrained optimization and fixed parameters.

Usage

maxBFGS(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ... )

maxCG(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)), ...)

maxSANN(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ... )

maxNM(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ...)

Arguments

fn	function to be maximised. Must have the parameter vector as the first argument. In order to use numeric gradient and BHHH method, fn must return a vector of observation-specific likelihood values. Those are summed internally where necessary. If the parameters are out of range, fn should return NA. See details for constant parameters.
grad	gradient of fn. Must have the parameter vector as the first argument. If NULL, numeric gradient is used (maxNM and maxSANN do not use gradient). Gradient may return a matrix, where columns correspond to the parameters and rows to the observations (useful for maxBHHH). The columns are summed internally.
hess	Hessian of fn. Not used by any of these methods, included for compatibility with maxNR.
start	initial values for the parameters. If start values are named, those names are also carried over to the results.
fixed	parameters to be treated as constants at their start values. If present, it is treated as an index vector of start parameters.
control	list of control parameters or a ‘MaxControl’ object. If it is a list, the default values are used for the parameters that are left unspecified by the user. These functions accept the following parameters: reltol sqrt(.Machine$double.eps), stopping condition. Relative convergence tolerance: the algorithm stops if the relative improvement between iterations is less than ‘reltol’. Note: for compatibility reason ‘tol’ is equivalent to ‘reltol’ for optim-based optimizers. iterlim integer, maximum number of iterations. Default values are 200 for ‘BFGS’, 500 (‘CG’ and ‘NM’), and 10000 (‘SANN’). Note that ‘iteration’ may mean different things for different optimizers. printLevel integer, larger number prints more working information. Default 0, no information. nm_alpha 1, Nelder-Mead simplex method reflection coefficient (see Nelder & Mead, 1965) nm_beta 0.5, Nelder-Mead contraction coefficient nm_gamma 2, Nelder-Mead expansion coefficient sann_cand NULL or a function for "SANN" algorithm to generate a new candidate point; if NULL, Gaussian Markov kernel is used (see argument gr of optim). sann_temp 10, starting temperature for the “SANN” cooling schedule. See optim. sann_tmax 10, number of function evaluations at each temperature for the “SANN” optimizer. See optim. sann_randomSeed 123, integer to seed random numbers to ensure replicability of “SANN” optimization and preserve R random numbers. Use options like sann_randomSeed=Sys.time() or sann_randomSeed=sample(100,1) if you want stochastic results.
constraints	either NULL for unconstrained optimization or a list with two components. The components may be either eqA and eqB for equality-constrained optimization A \theta + B = 0; or ineqA and ineqB for inequality constraints A \theta + B > 0. More than one row in ineqA and ineqB corresponds to more than one linear constraint, in that case all these must be zero (equality) or positive (inequality constraints). The equality-constrained problem is forwarded to sumt, the inequality-constrained case to constrOptim2.
finalHessian	how (and if) to calculate the final Hessian. Either FALSE (not calculate), TRUE (use analytic/numeric Hessian) or "bhhh"/"BHHH" for information equality approach. The latter approach is only suitable for maximizing log-likelihood function. It requires the gradient/log-likelihood to be supplied by individual observations, see maxBHHH for details.
parscale	A vector of scaling values for the parameters. Optimization is performed on 'par/parscale' and these should be comparable in the sense that a unit change in any element produces about a unit change in the scaled value. (see optim)
...	further arguments for fn and grad.

Details

In order to provide a consistent interface, all these functions also accept arguments that other optimizers use. For instance, maxNM accepts the ‘grad’ argument despite being a gradient-less method.

The ‘state’ (or ‘seed’) of R's random number generator is saved at the beginning of the maxSANN function and restored at the end of this function so this function does not affect the generation of random numbers although the random seed is set to argument random.seed and the ‘SANN’ algorithm uses random numbers.

Value

object of class "maxim". Data can be extracted through the following functions:

maxValue	fn value at maximum (the last calculated value if not converged.)
coef	estimated parameter value.
gradient	vector, last calculated gradient value. Should be close to 0 in case of normal convergence.
estfun	matrix of gradients at parameter value estimate evaluated at each observation (only if grad returns a matrix or grad is not specified and fn returns a vector).
hessian	Hessian at the maximum (the last calculated value if not converged).
returnCode	integer. Success code, 0 is success (see optim).
returnMessage	a short message, describing the return code.
activePar	logical vector, which parameters are optimized over. Contains only TRUE-s if no parameters are fixed.
nIter	number of iterations. Two-element integer vector giving the number of calls to fn and gr, respectively. This excludes those calls needed to compute the Hessian, if requested, and any calls to fn to compute a finite-difference approximation to the gradient.
maximType	character string, type of maximization.
maxControl	the optimization control parameters in the form of a MaxControl object.

The following components can only be extracted directly (with \$):

constraints

A list, describing the constrained optimization (NULL if unconstrained). Includes the following components: type type of constrained optimization outer.iterations number of iterations in the constraints step barrier.value value of the barrier function

Author(s)

Ott Toomet, Arne Henningsen

References

Nelder, J. A. & Mead, R. A, Simplex Method for Function Minimization, The Computer Journal, 1965, 7, 308-313

See Also

optim, nlm, maxNR, maxBHHH, maxBFGSR for a maxNR-based BFGS implementation.

Examples

# Maximum Likelihood estimation of Poissonian distribution
n <- rpois(100, 3)
loglik <- function(l) n*log(l) - l - lfactorial(n)
# we use numeric gradient
summary(maxBFGS(loglik, start=1))
# you would probably prefer mean(n) instead of that ;-)
# Note also that maxLik is better suited for Maximum Likelihood
###
### Now an example of constrained optimization
###
f <- function(theta) {
  x <- theta[1]
  y <- theta[2]
  exp(-(x^2 + y^2))
  ## you may want to use exp(- theta %*% theta) instead
}
## use constraints: x + y >= 1
A <- matrix(c(1, 1), 1, 2)
B <- -1
res <- maxNM(f, start=c(1,1), constraints=list(ineqA=A, ineqB=B),
control=list(printLevel=1))
print(summary(res))

maxLik documentation built on May 29, 2024, 2:32 a.m.