bfgs: Broyden-Fletcher-Goldfarb-Shanno (BFGS) Optimization

In optimflex: Derivative-Based Optimization with User-Defined Convergence Criteria

Broyden-Fletcher-Goldfarb-Shanno (BFGS) Optimization

Description

Implements the damped BFGS Quasi-Newton algorithm with a Strong Wolfe line search for non-linear optimization, specifically tailored for SEM.

Usage

bfgs(
  start,
  objective,
  gradient = NULL,
  hessian = NULL,
  lower = -Inf,
  upper = Inf,
  control = list(),
  ...
)

Arguments

start	Numeric vector. Starting values for the optimization parameters.
objective	Function. The objective function to minimize.
gradient	Function (optional). Gradient of the objective function.
hessian	Function (optional). Hessian matrix of the objective function.
lower	Numeric vector. Lower bounds for box constraints.
upper	Numeric vector. Upper bounds for box constraints.
control	List. Control parameters including convergence flags: use_abs_f: Logical. Use absolute change in objective for convergence. use_rel_f: Logical. Use relative change in objective for convergence. use_abs_x: Logical. Use absolute change in parameters for convergence. use_rel_x: Logical. Use relative change in parameters for convergence. use_grad: Logical. Use gradient norm for convergence. use_posdef: Logical. Verify positive definiteness at convergence. use_pred_f: Logical. Record predicted objective decrease. use_pred_f_avg: Logical. Record average predicted decrease. diff_method: String. Method for numerical differentiation.
...	Additional arguments passed to objective, gradient, and Hessian functions.

Details

bfgs is a Quasi-Newton method that maintains an approximation of the inverse Hessian matrix. It is widely considered the most robust and efficient member of the Broyden family of optimization methods.

BFGS vs. DFP: While both bfgs and dfp update the inverse Hessian using rank-two formulas, BFGS is generally more tolerant of inaccuracies in the line search. This implementation uses the Sherman-Morrison formula to update the inverse Hessian directly, avoiding the need for matrix inversion at each step.

Strong Wolfe Line Search: To maintain the positive definiteness of the Hessian approximation and ensure global convergence, this algorithm employs a Strong Wolfe line search. This search identifies a step length \alpha that satisfies both sufficient decrease (Armijo condition) and the curvature condition.

Damping for Non-Convexity: In Structural Equation Modeling (SEM), objective functions often exhibit non-convex regions. When use_damped = TRUE, Powell's damping strategy is applied to the update vectors to preserve the positive definiteness of the Hessian approximation even when the curvature condition is not naturally met.

Value

A list containing optimization results and iteration metadata.

References

• Nocedal, J., & Wright, S. J. (2006). Numerical Optimization. Springer.

• Fletcher, R. (1987). Practical Methods of Optimization. Wiley.

Examples

quad <- function(x) (x[1] - 2)^2 + (x[2] + 1)^2
res <- bfgs(start = c(0, 0), objective = quad)
print(res$par)

optimflex documentation built on April 11, 2026, 5:06 p.m.