R/funconstrain.R

In jlmelville/funconstrain: Functions for Testing Unconstrained Numerical Optimization

#' funconstrain: Test functions for unconstrained minimization.
#'
#' The funconstrain package provides 35 test functions taken from the paper
#' of More', Garbow and Hillstrom, useful for testing numerical optimization
#' methods.
#'
#' The functions all take the form of a nonlinear least squares problem, with
#' the goal of minimizing the output of \code{m} functions, each of which is
#' a function, \eqn{f_i}{fi} of the same \code{n} parameters:
#'
#' \deqn{f\left(\left[x_1, x_2, \ldots, x_n \right ]\right) =
#' \sum_{i = 1}^{m}f_{i}^2\left(\left[x_1, x_2, \ldots, x_n \right ]\right)}{
#' f([x1, x2, ..., xn]) = sum(i = 1:m) fi([x1, x2, ..., xn]) ^ 2}
#'
#' @section Function Details:
#'
#' The documentation of each function provides details on:
#' \itemize{
#'   \item \code{m} The number of summand functions, \eqn{f_i}{fi}. Some
#'   functions are defined for a fixed \code{m}, others require the user to
#'   specify a value of \code{m}.
#'   \item \code{n} The numer of parameters. Some functions are defined for a
#'   fixed value of \code{n}, others allow different values of \code{n} to be
#'   specified. This is done by passing a vector of parameters of the desired
#'   length to the objective function.
#'   \item The values and locations of minima. Some functions have multiple
#'   minima, which can make them less desirable for comparing methods. Some
#'   functions have only the value of the objective function at the minima
#'   specified, and not the location.
#' }
#'
#' @section Test Function Parameters:
#'
#' Most numerical optimization methods use both the objective value and the
#' gradient of the value with respect to the parameters. So for a given test
#' problem, you will want both the objective function itself, a function to
#' calculate the gradient and a place to start the optimization from. The
#' functions provided by funconstrain are therefore not used directly in
#' an optimization method. They are factory functions which will generate
#' the objective and gradient functions you want.
#'
#' The test problems are all generated by calling the desired function. Some
#' functions have the following parameter:
#'
#' \itemize{
#'   \item \code{m} The number of summand functions as described above. This
#'   needs only to be provided when the test problem allows for variable
#'   \code{m}. Default values are provided in all these cases, but be aware
#'   that the defaults were chosen abritrarily, and some functions put
#'   a restriction on the acceptable value of \code{m} based on the number
#'   of parameters, \code{n}. If in doubt, ensure it's specified explicitly.
#' }
#'
#' @section Test Function Return Value:
#'
#' The return value is a list containing:
#'
#' \itemize{
#'   \item \code{fn} The objective function. This takes a numeric vector of
#'   length \code{n} representing the set of parameters to be optimized. It
#'   returns a scalar, the value of the objective function.
#'   \item \code{gr} The gradient function of the objective. This takes a
#'   numeric vector of length \code{n} representing the set of parameters to be
#'   optimized. It returns a vector also of length \code{n}, representing the
#'   gradient of the objective function with respect to the \code{n} parameters.
#'   \item \code{fg} A function which calculates both the objective function and
#'   the gradient in one call. This takes a numeric vector of length \code{n}
#'   representing the set of parameters to be optimized. It returns a list
#'   with two members: \code{fn}, the scalar objective function; and \code{gr},
#'   the gradient vector. This is a convenience function: not all optimization
#'   methods can make use of it, but for those that can, it is common to require
#'   both the function and gradient for a given set of parameters, and there is
#'   often a lot of shared calculations that can make calculating both at the
#'   same time more efficient than calling the \code{fn} and \code{gr} functions
#'   separately.
#'   \item \code{x0} A suggested starting location. For those functions where
#'   the number of parameters, \code{n} is fixed, this is a fixed length
#'   numeric vector. Where \code{n} can have multiple values, this is a function
#'   which takes one value, \code{n} and returns a numeric vector of length
#'   \code{n}. Where \code{x0} is a function, \code{n} also has a default. Like
#'   \code{m}, these have been chosen arbitrarily, but with the idea that if
#'   you were to use these test functions in the order provided in the MGH
#'   paper, the value of \code{n} increases from \code{2} to \code{50}.
#'   \item \code{fmin} the reported minimum function value
#'   \item \code{xmin} a numeric vector with the reported minimum parameters
#' }
#'
#' It's much more straightforward than it sounds. See the 'Examples' section
#' below or the examples in each function.
#'
#' @section Available Functions:
#'
#' The names of the test functions are given below, using the same numbering as
#' in the original More, Garbow and Hillstrom paper:
#'
#' \enumerate{
#'   \item \code{\link{rosen}} Rosenbrock function.
#'   \item \code{\link{freud_roth}} Freudenstein and Roth function.
#'   \item \code{\link{powell_bs}} Powell badly scaled function.
#'   \item \code{\link{brown_bs}} Brown badly scaled function.
#'   \item \code{\link{beale}} Beale function.
#'   \item \code{\link{jenn_samp}} Jennrich and Sampson function.
#'   \item \code{\link{helical}} Helical valley function.
#'   \item \code{\link{bard}} Bard function.
#'   \item \code{\link{gauss}} Gaussian function.
#'   \item \code{\link{meyer}} Meyer function.
#'   \item \code{\link{gulf}} Gulf research and development function.
#'   \item \code{\link{box_3d}} Box three-dimensional function.
#'   \item \code{\link{powell_s}} Powell singular function.
#'   \item \code{\link{wood}} Wood function.
#'   \item \code{\link{kow_osb}} Kowalik and Osborne function.
#'   \item \code{\link{brown_den}} Brown and Dennis function.
#'   \item \code{\link{osborne_1}} Osborne 1 function.
#'   \item \code{\link{biggs_exp6}} Biggs EXP6 function.
#'   \item \code{\link{osborne_2}} Osborne 2 function.
#'   \item \code{\link{watson}} Watson function.
#'   \item \code{\link{ex_rosen}} Extended Rosenbrock function.
#'   \item \code{\link{ex_powell}} Extended Powell function.
#'   \item \code{\link{penalty_1}} Penalty function I.
#'   \item \code{\link{penalty_2}} Penalty function II.
#'   \item \code{\link{var_dim}} Variable dimensioned function.
#'   \item \code{\link{trigon}} Trigonometric function.
#'   \item \code{\link{brown_al}} Brown almost-linear function.
#'   \item \code{\link{disc_bv}} Discrete boundary value function.
#'   \item \code{\link{disc_ie}} Discrete integral equation function.
#'   \item \code{\link{broyden_tri}} Broyden tridiagonal function.
#'   \item \code{\link{broyden_band}} Broyden banded function.
#'   \item \code{\link{linfun_fr}} Linear function - full rank.
#'   \item \code{\link{linfun_r1}} Linear function - rank 1.
#'   \item \code{\link{linfun_r1z}} Linear function - rank 1 with zero columns
#'   and rows.
#'   \item \code{\link{chebyquad}} Chebyquad function.
#' }
#'
#' For details, see the specific function help text.
#'
#' @references
#' More', J. J., Garbow, B. S., & Hillstrom, K. E. (1981).
#' Testing unconstrained optimization software.
#' \emph{ACM Transactions on Mathematical Software (TOMS)}, \emph{7}(1), 17-41.
#' \doi{doi.org/10.1145/355934.355936}
#'
#' @examples
#'
#' # Fixed m and n
#' # The famous Rosenbrock function has fixed m and n (2 in each case)
#' rbrock <- rosen()
#' # Pass the objective function, gradient and starting point to an optimization
#' # method:
#' res <- stats::optim(par = rbrock$x0, fn = rbrock$fn, gr = rbrock$gr,
#' method = "L-BFGS-B")
#' # Or feel free to ignore the suggested starting point and use your own:
#' res <- stats::optim(par = c(1.2, 1.2), fn = rbrock$fn, gr = rbrock$gr,
#' method = "L-BFGS-B")
#'
#' # Multiple m, fixed n
#' # The gulf test problem allows for multiple m:
#' gulf_m10 <- gulf(m = 10)
#' res_m10 <- stats::optim(par = gulf_m10$x0, fn = gulf_m10$fn, gr =
#' gulf_m10$gr, method = "L-BFGS-B")
#'
#' # Using a different m will give different results, although in the case
#' # of the gulf problem you reach the same minimum.
#' gulf_m20 <- gulf(m = 20)
#' res_m20 <- stats::optim(par = gulf_m20$x0, fn = gulf_m20$fn, gr =
#' gulf_m20$gr, method = "L-BFGS-B")
#'
#' # Fixed m, multiple n
#' # The Chebyquad function is defined for variable values of n, but the value
#' # of m is fixed
#' cheby <- chebyquad()
#'
#' # To use different values of n, we provide it to the starting point x0, which
#' # is a function when n can take multiple values.
#' # A five-parameter version:
#' res_n5 <- stats::optim(par = cheby$x0(n = 5), fn = cheby$fn, gr = cheby$gr,
#' method = "L-BFGS-B")
#' # And a 10-parameter version:
#' res_n10 <- stats::optim(par = cheby$x0(n = 10), fn = cheby$fn, gr = cheby$gr,
#' method = "L-BFGS-B")
#'
#' # Multiple m, multiple n
#' # The linear function full rank function requires both m and n to be
#' # specified:
#' lf_m10 <- linfun_fr(m = 10)
#'
#' # The n = 10, m = 10 solution:
#' res_m10_n10 <- stats::optim(par = lf_m10$x0(n = 10), fn = lf_m10$fn, gr =
#' lf_m10$gr, method = "L-BFGS-B")
#'
#' # The n = 5, m = 10 solution:
#' res_m10_n5 <- stats::optim(par = lf_m10$x0(n = 5), fn = lf_m10$fn, gr =
#' lf_m10$gr, method = "L-BFGS-B")
#'
#' # Repeat the above, but this time with m = 20
#' lf_m20 <- linfun_fr(m = 20)
#'
#' # The n = 10, m = 20 solution:
#' res_m20_n10 <- stats::optim(par = lf_m20$x0(n = 10), fn = lf_m20$fn, gr =
#' lf_m10$gr, method = "L-BFGS-B")
#'
#' # The n = 5, m = 20 solution:
#' res_m20_n5 <- stats::optim(par = lf_m20
#' $x0(n = 5), fn = lf_m20$fn, gr =
#' lf_m10$gr, method = "L-BFGS-B")
#'
#' @name funconstrain
"_PACKAGE"