maxBFGS: BFGS, conjugate gradient, SANN and Nelder-Mead Maximization
In maxLik: Maximum Likelihood Estimation and Related Tools

Description Usage Arguments Details Value Author(s) References See Also Examples

These functions are wrappers for optim, adding constrained optimization and fixed parameters.

maxBFGS(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ... )

maxCG(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)), ...)

maxSANN(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ... )

maxNM(fn, grad=NULL, hess=NULL, start, fixed=NULL,
   control=NULL,
   constraints=NULL,
   finalHessian=TRUE,
   parscale=rep(1, length=length(start)),
   ...)

`fn`	function to be maximised. Must have the parameter vector as the first argument. In order to use numeric gradient and BHHH method, `fn` must return a vector of observation-specific likelihood values. Those are summed internally where necessary. If the parameters are out of range, `fn` should return `NA`. See details for constant parameters.
`grad`	gradient of `fn`. Must have the parameter vector as the first argument. If `NULL`, numeric gradient is used (`maxNM` and `maxSANN` do not use gradient). Gradient may return a matrix, where columns correspond to the parameters and rows to the observations (useful for maxBHHH). The columns are summed internally.
`hess`	Hessian of `fn`. Not used by any of these methods, included for compatibility with `maxNR`.
`start`	initial values for the parameters. If start values are named, those names are also carried over to the results.
`fixed`	parameters to be treated as constants at their `start` values. If present, it is treated as an index vector of `start` parameters.
`control`	list of control parameters or a ‘MaxControl’ object. If it is a list, the default values are used for the parameters that are left unspecified by the user. These functions accept the following parameters: reltol sqrt(.Machine$double.eps), stopping condition. Relative convergence tolerance: the algorithm stops if the relative improvement between iterations is less than ‘reltol’. Note: for compatibility reason ‘tol’ is equivalent to ‘reltol’ for optim-based optimizers. iterlim integer, maximum number of iterations. Default values are 200 for ‘BFGS’, 500 (‘CG’ and ‘NM’), and 10000 (‘SANN’). Note that ‘iteration’ may mean different things for different optimizers. printLevel integer, larger number prints more working information. Default 0, no information. nm_alpha 1, Nelder-Mead simplex method reflection coefficient (see Nelder & Mead, 1965) nm_beta 0.5, Nelder-Mead contraction coefficient nm_gamma 2, Nelder-Mead expansion coefficient sann_cand `NULL` or a function for `"SANN"` algorithm to generate a new candidate point; if `NULL`, Gaussian Markov kernel is used (see argument `gr` of `optim`). sann_temp 10, starting temperature for the “SANN” cooling schedule. See `optim`. sann_tmax 10, number of function evaluations at each temperature for the “SANN” optimizer. See `optim`. sann_randomSeed 123, integer to seed random numbers to ensure replicability of “SANN” optimization and preserve `R` random numbers. Use options like `sann_randomSeed=Sys.time()` or `sann_randomSeed=sample(100,1)` if you want stochastic results.
`constraints`	either `NULL` for unconstrained optimization or a list with two components. The components may be either `eqA` and `eqB` for equality-constrained optimization A %% theta + B = 0; or `ineqA` and `ineqB` for inequality constraints A %% theta + B > 0. More than one row in `ineqA` and `ineqB` corresponds to more than one linear constraint, in that case all these must be zero (equality) or positive (inequality constraints). The equality-constrained problem is forwarded to `sumt`, the inequality-constrained case to `constrOptim2`.
`finalHessian`	how (and if) to calculate the final Hessian. Either `FALSE` (not calculate), `TRUE` (use analytic/numeric Hessian) or `"bhhh"`/`"BHHH"` for information equality approach. The latter approach is only suitable for maximizing log-likelihood function. It requires the gradient/log-likelihood to be supplied by individual observations, see `maxBHHH` for details.
`parscale`	A vector of scaling values for the parameters. Optimization is performed on 'par/parscale' and these should be comparable in the sense that a unit change in any element produces about a unit change in the scaled value. (see `optim`)
`...`	further arguments for `fn` and `grad`.

In order to provide a consistent interface, all these functions also accept arguments that other optimizers use. For instance, maxNM accepts the ‘grad’ argument despite being a gradient-less method.

The ‘state’ (or ‘seed’) of R's random number generator is saved at the beginning of the maxSANN function and restored at the end of this function so this function does not affect the generation of random numbers although the random seed is set to argument random.seed and the ‘SANN’ algorithm uses random numbers.

object of class "maxim". Data can be extracted through the following functions:

`maxValue`	`fn` value at maximum (the last calculated value if not converged.)
`coef`	estimated parameter value.
`gradient`	vector, last calculated gradient value. Should be close to 0 in case of normal convergence.
`estfun`	matrix of gradients at parameter value `estimate` evaluated at each observation (only if `grad` returns a matrix or `grad` is not specified and `fn` returns a vector).
`hessian`	Hessian at the maximum (the last calculated value if not converged).
`returnCode`	integer. Success code, 0 is success (see `optim`).
`returnMessage`	a short message, describing the return code.
`activePar`	logical vector, which parameters are optimized over. Contains only `TRUE`-s if no parameters are fixed.
`nIter`	number of iterations. Two-element integer vector giving the number of calls to `fn` and `gr`, respectively. This excludes those calls needed to compute the Hessian, if requested, and any calls to `fn` to compute a finite-difference approximation to the gradient.
`maximType`	character string, type of maximization.
`maxControl`	the optimization control parameters in the form of a `MaxControl` object.

The following components can only be extracted directly (with \$):

constraints

A list, describing the constrained optimization (NULL if unconstrained). Includes the following components:

type: type of constrained optimization
outer.iterations: number of iterations in the constraints step
barrier.value: value of the barrier function

Ott Toomet, Arne Henningsen

Nelder, J. A. & Mead, R. A, Simplex Method for Function Minimization, The Computer Journal, 1965, 7, 308-313

optim, nlm, maxNR, maxBHHH, maxBFGSR for a maxNR-based BFGS implementation.

# Maximum Likelihood estimation of Poissonian distribution
n <- rpois(100, 3)
loglik <- function(l) n*log(l) - l - lfactorial(n)
# we use numeric gradient
summary(maxBFGS(loglik, start=1))
# you would probably prefer mean(n) instead of that ;-)
# Note also that maxLik is better suited for Maximum Likelihood
###
### Now an example of constrained optimization
###
f <- function(theta) {
  x <- theta[1]
  y <- theta[2]
  exp(-(x^2 + y^2))
  ## you may want to use exp(- theta %*% theta) instead
}
## use constraints: x + y >= 1
A <- matrix(c(1, 1), 1, 2)
B <- -1
res <- maxNM(f, start=c(1,1), constraints=list(ineqA=A, ineqB=B),
control=list(printLevel=1))
print(summary(res))

Loading required package: miscTools

Please cite the 'maxLik' package as:
Henningsen, Arne and Toomet, Ott (2011). maxLik: A package for maximum likelihood estimation in R. Computational Statistics 26(3), 443-458. DOI 10.1007/s00180-010-0217-1.

If you have questions, suggestions, or comments regarding the 'maxLik' package, please use a forum or 'tracker' at maxLik's R-Forge site:
https://r-forge.r-project.org/projects/maxlik/
--------------------------------------------
BFGS maximization 
Number of iterations: 30 
Return code: 0 
successful convergence  
Function value: -190.6299 
Estimates:
     estimate     gradient
[1,] 3.149999 1.680878e-05
--------------------------------------------
  Nelder-Mead direct search function minimizer
function value for initial parameters = -0.135135
  Scaled convergence tolerance is 2.01367e-09
Stepsize computed as 0.100000
BUILD              3 -0.109500 -0.135135
LO-REDUCTION       5 -0.109500 -0.135135
EXTENSION          7 -0.132455 -0.196709
LO-REDUCTION       9 -0.135135 -0.196709
EXTENSION         11 -0.196709 -0.375079
LO-REDUCTION      13 -0.196709 -0.375079
HI-REDUCTION      15 -0.276440 -0.375079
REFLECTION        17 -0.367648 -0.484044
LO-REDUCTION      19 -0.375079 -0.484044
HI-REDUCTION      21 -0.429307 -0.484044
REFLECTION        23 -0.484044 -0.545734
LO-REDUCTION      25 -0.484044 -0.545734
HI-REDUCTION      27 -0.513326 -0.545734
REFLECTION        29 -0.534921 -0.572951
REFLECTION        31 -0.545734 -0.572951
HI-REDUCTION      33 -0.560758 -0.572951
REFLECTION        35 -0.572951 -0.590899
LO-REDUCTION      37 -0.572951 -0.590899
REFLECTION        39 -0.582560 -0.601943
HI-REDUCTION      41 -0.590442 -0.601943
LO-REDUCTION      43 -0.590899 -0.601943
HI-REDUCTION      45 -0.596296 -0.601943
HI-REDUCTION      47 -0.598979 -0.601943
REFLECTION        49 -0.600631 -0.604251
LO-REDUCTION      51 -0.601943 -0.604251
HI-REDUCTION      53 -0.603278 -0.604251
HI-REDUCTION      55 -0.603935 -0.604251
REFLECTION        57 -0.604160 -0.605162
HI-REDUCTION      59 -0.604251 -0.605162
HI-REDUCTION      61 -0.604694 -0.605162
LO-REDUCTION      63 -0.604718 -0.605162
HI-REDUCTION      65 -0.604949 -0.605162
REFLECTION        67 -0.605080 -0.605354
HI-REDUCTION      69 -0.605162 -0.605354
LO-REDUCTION      71 -0.605207 -0.605354
REFLECTION        73 -0.605318 -0.605454
LO-REDUCTION      75 -0.605354 -0.605454
HI-REDUCTION      77 -0.605409 -0.605454
LO-REDUCTION      79 -0.605422 -0.605454
LO-REDUCTION      81 -0.605452 -0.605459
HI-REDUCTION      83 -0.605454 -0.605459
LO-REDUCTION      85 -0.605458 -0.605459
HI-REDUCTION      87 -0.605459 -0.605459
HI-REDUCTION      89 -0.605459 -0.605459
LO-REDUCTION      91 -0.605459 -0.605460
HI-REDUCTION      93 -0.605459 -0.605460
HI-REDUCTION      95 -0.605460 -0.605460
LO-REDUCTION      97 -0.605460 -0.605460
HI-REDUCTION      99 -0.605460 -0.605460
HI-REDUCTION     101 -0.605460 -0.605460
HI-REDUCTION     103 -0.605460 -0.605460
Exiting from Nelder Mead minimizer
    105 function evaluations used
--------------------------------------------
Nelder-Mead maximization 
Number of iterations: 105 
Return code: 0 
successful convergence  
Function value: 0.6064308 
Estimates:
      estimate   gradient
[1,] 0.5001167 -0.6065724
[2,] 0.5000479 -0.6064889

Constrained optimization based on constrOptim 
1  outer iterations, barrier value -0.0009712347 
--------------------------------------------