gauss_newton: Gauss-Newton Optimization

In optimflex: Derivative-Based Optimization with User-Defined Convergence Criteria

Gauss-Newton Optimization

Description

Implements a full-featured Gauss-Newton algorithm for non-linear optimization, specifically optimized for Structural Equation Modeling (SEM).

Usage

gauss_newton(
  start,
  objective,
  residual = NULL,
  gradient = NULL,
  hessian = NULL,
  jac = NULL,
  lower = -Inf,
  upper = Inf,
  control = list(),
  ...
)

Arguments

start	Numeric vector. Starting values for the optimization parameters.
objective	Function. The objective function to minimize.
residual	Function (optional). Function that returns the residuals vector.
gradient	Function (optional). Gradient of the objective function.
hessian	Function (optional). Hessian matrix of the objective function.
jac	Function (optional). Jacobian matrix of the residuals.
lower	Numeric vector. Lower bounds for box constraints.
upper	Numeric vector. Upper bounds for box constraints.
control	List. Control parameters including convergence flags: use_abs_f: Logical. Use absolute change in objective for convergence. use_rel_f: Logical. Use relative change in objective for convergence. use_abs_x: Logical. Use absolute change in parameters for convergence. use_rel_x: Logical. Use relative change in parameters for convergence. use_grad: Logical. Use gradient norm for convergence. use_posdef: Logical. Verify positive definiteness at convergence. use_pred_f: Logical. Record predicted objective decrease. use_pred_f_avg: Logical. Record average predicted decrease. diff_method: String. Method for numerical differentiation.
...	Additional arguments passed to objective, gradient, and Hessian functions.

Details

gauss_newton is a specialized optimization algorithm for least-squares and Maximum Likelihood problems where the objective function can be expressed as a sum of squared residuals.

Scaling and SEM Consistency: To ensure consistent simulation results and standard error (SE) calculations, this implementation adjusts the Gradient (2J^T r) and the Approximate Hessian (2J^T J) to match the scale of the Maximum Likelihood (ML) fitting function F_{ML}. This alignment is critical when calculating asymptotic covariance matrices using the formula \frac{2}{n} H^{-1}.

Comparison with Newton-Raphson: Unlike newton_raphson or modified_newton, which require the full second-order Hessian, Gauss-Newton approximates the Hessian using the Jacobian of the residuals. This is computationally more efficient and provides a naturally positive-semidefinite approximation, though a ridge adjustment is still provided for numerical stability.

Ridge Adjustment Strategy: The function includes a "Ridge Rescue" mechanism. If the approximate Hessian is singular or poorly conditioned for Cholesky decomposition, it iteratively adds a diagonal ridge (\tau I) until numerical stability is achieved.

Value

A list containing optimization results and iteration metadata.

References

• Nocedal, J., & Wright, S. J. (2006). Numerical Optimization. Springer.

• Bollen, K. A. (1989). Structural Equations with Latent Variables. Wiley.

Examples

quad <- function(x) (x[1] - 2)^2 + (x[2] + 1)^2
res <- gauss_newton(start = c(0, 0), objective = quad)
print(res$par)

optimflex documentation built on April 11, 2026, 5:06 p.m.