adaptive_gd: Adaptive gradient descent

In ngme2: Linear Latent Non-Gaussian Models with Flexible Distributions

Adaptive gradient descent

Description

Adaptive gradient descent optimizer.

Usage

adaptive_gd(stepsize = 0.01)

Arguments

stepsize

initial stepsize for SGD

Details

Based on the method described in https://arxiv.org/pdf/1910.09529. The update rule for adaptive gradient descent is:

\lambda_k = \min(\sqrt{1 + \theta_{k-1}} \lambda_{k-1}, \frac{||x_k - x_{k-1}||}{2 ||\nabla f(x_k) - \nabla f(x_{k-1})||} )

x_{k+1} = x_k - \lambda_k \nabla f(x_k)

\theta_k = \lambda_k / \lambda_{k-1}

Value

a list of control variables for optimization (used in control_opt function)

ngme2 documentation built on May 20, 2026, 9:10 a.m.