adam: Optimize mathematical function using the Adam algorithm

In PhilippScheller/visualDescend: Visulaization of descent method for multi-dimensional optimization functions

Description

This functions uses the Adam algorithm to find the minimum of a (multi-) dimensional mathematical function. The combination considers both, the average of the previous gradients (Momentum Optimizer) and the average of the square gradients (RMS Prop), both under exponential decay.

Usage

adam(
  f,
  x0,
  max.iter = 100,
  step.size = 0.1,
  phi1 = 0.5,
  phi2 = 0.8,
  stop.grad = .Machine$double.eps
)

Arguments

f	a (multi-) dimensional function to be eptimized.
x0	the starting point of the optimization.
max.iter	the maximum number of iterations performed in the optimization.
step.size	the step size (sometimes referred to as 'learn-rate') of the optimization.
phi1	decay rate for RMS Prop term, i.e. the squared gradients.
phi2	decay rate for Momentum term, i.e. the previous gradients.
stop.grad	the stop-criterion for the gradient change.

PhilippScheller/visualDescend documentation built on Feb. 5, 2020, 4:04 a.m.