lr_scheduler_cyclic: Cyclical learning rate scheduler

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Cyclical learning rate scheduler

Description

Cycles the learning rate linearly between base_lr and max_lr over a triangular cycle of 2 * step_size_up steps (Smith, 2017). Each call to $step() advances one step — call it per batch, not per epoch.

Usage

lr_scheduler_cyclic(
  optimizer,
  base_lr,
  max_lr,
  step_size_up = 2000L,
  step_size_down = NULL,
  mode = c("triangular", "triangular2", "exp_range"),
  gamma = 1
)

Arguments

optimizer	Optimizer environment.
base_lr	Lower learning rate bound.
max_lr	Upper learning rate bound.
step_size_up	Steps in the increasing half of a cycle (default 2000).
step_size_down	Steps in the decreasing half. NULL (default) mirrors step_size_up.
mode	One of "triangular" (default), "triangular2" (amplitude halved each cycle) or "exp_range" (amplitude scaled by gamma^step).
gamma	Decay factor for mode = "exp_range" (default 1.0).

Value

An lr_scheduler_cyclic environment with $step() and $get_lr().

References

Smith, L. N. (2017). Cyclical Learning Rates for Training Neural Networks. WACV.

Examples

w   <- ag_param(matrix(runif(4), 2, 2))
opt <- optimizer_sgd(list(w = w), lr = 0.01)
sch <- lr_scheduler_cyclic(opt, base_lr = 0.001, max_lr = 0.01,
                           step_size_up = 10L)
for (i in 1:40) sch$step()

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.