adam: ADAM stochastic process update

In jtipton25/sgMRA: Multi-Resolution Gaussian Process Approximations with Stochastic Gradient Descent

ADAM stochastic process update

Description

ADAM stochastic process update

Usage

adam(k, grad, m, v, beta1 = 0.9, beta2 = 0.999, epsilon = 1e-08)

Arguments

k	The gradient descent iteration
grad	A list of gradient vectors
m	A list of momentums
v	A list of velocities
beta1	The decay rate for the momentum
beta2	The decay rate for the velocity
epsilon	A small amount to prevent a divide-by-zero error

jtipton25/sgMRA documentation built on Feb. 9, 2023, 4:53 a.m.