dot-damped_bfgs: Damped BFGS Optimizer for Penalty Tuning

In lgspline: Lagrangian Multiplier Smoothing Splines for Smooth Function Estimation

Damped BFGS Optimizer for Penalty Tuning

Description

Custom implementation of damped BFGS quasi-Newton optimization for minimizing the selected tuning criterion. Uses step-size damping with backtracking and Sherman-Morrison-Woodbury inverse Hessian updates.

Usage

.damped_bfgs(
  par,
  log_penalty_vec,
  criterion_fxn,
  gr_fxn,
  env,
  tol,
  parallel_bfgs = TRUE,
  max_iter = 100,
  ...
)

Arguments

par	Numeric vector; initial log-scale penalty parameters (first two elements are log(wiggle) and log(flat_ridge)).
log_penalty_vec	Numeric vector; log-scale predictor/partition penalties appended to the optimization vector.
criterion_fxn	Function; tuning-objective evaluation function with signature function(par, log_penalty_vec, env, ...).
gr_fxn	Function; gradient function with signature function(par, log_penalty_vec, outlist, env, ...).
env	List; tuning environment (passed through to criterion_fxn and gr_fxn).
tol	Numeric; convergence tolerance for parameter change and the strict absolute criterion-change check. A small scale-aware plateau check is also used after the first few iterations.
max_iter	Integer; maximum number of BFGS iterations (default 100).
...	Additional arguments passed to fitting functions.

Details

The optimizer uses the following strategy:

1. Iterations 1-2: steepest descent with damping.

2. Iteration 3+: BFGS quasi-Newton with inverse Hessian approximation updated via the standard secant condition. Falls back to identity matrix when the update is numerically unstable.

3. Step acceptance: Armijo-like criterion (accept if \mathrm{GCV}_{u}^{(\mathrm{new})} \leq \mathrm{GCV}_{u}^{(\mathrm{old})}).

4. Backtracking: damping factor halved on rejection; terminates when damp < 2^{-10} (early iterations) or 2^{-12} (later iterations).

5. Convergence: after iteration 9, stop when either the strict tol checks are met or the accepted criterion changes remain below a small criterion-scale tolerance for several iterations.

Value

List containing the best parameter vector found, the corresponding criterion value, and the number of iterations performed.

lgspline documentation built on May 8, 2026, 5:07 p.m.