optimizer_adagrad: Adagrad optimizer

In SPOTMisc: Misc Extensions for the 'SPOT' Package

Adagrad optimizer

Description

Adagrad optimizer as described in [Adaptive Subgradient Methods for Online Learning and Stochastic Optimization](https://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf).

Usage

optimizer_adagrad(
  learning_rate = 0.01,
  epsilon = NULL,
  decay = 0,
  clipnorm = NULL,
  clipvalue = NULL,
  ...
)

Arguments

learning_rate	float >= 0. Learning rate.
epsilon	float >= 0. Fuzz factor. If 'NULL', defaults to 'k_epsilon()'.
decay	float >= 0. Learning rate decay over each update.
clipnorm	Gradients will be clipped when their L2 norm exceeds this value.
clipvalue	Gradients will be clipped when their absolute value exceeds this value.
...	Unused, present only for backwards compatability

Note

To enable compatibility with the ranges of the learning rates of the other optimizers, the learning rate learning_rate is internally mapped to 10 * learning_rate. That is, a learning rat of 0.001 will be mapped to 0.01 (which is the default.)

See Also

Other optimizers: optimizer_adadelta(), optimizer_adamax(), optimizer_adam(), optimizer_nadam(), optimizer_rmsprop(), optimizer_sgd()

SPOTMisc documentation built on Sept. 5, 2022, 5:06 p.m.