optimizer_sgd: Create an SGD optimizer

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Create an SGD optimizer

Description

Create an SGD optimizer

Usage

optimizer_sgd(
  params,
  lr = 0.01,
  momentum = 0,
  accumulate_steps = 1L,
  average = TRUE
)

Arguments

params	List of ag_param tensors, named or not (names are used only for display). The same tensor listed twice is kept once, with a warning.
lr	Learning rate (default 0.01)
momentum	Momentum factor (default 0)
accumulate_steps	Number of backward() passes to accumulate before a call to $step() updates the weights (default 1, no accumulation). With k > 1 the first k - 1 calls to $step() only count, and the k-th applies the update and resets the counter, so a micro-batch loop needs no bookkeeping of its own.
average	When accumulating, divide the accumulated gradient by accumulate_steps (default TRUE). That is what makes k micro-batches equivalent to one batch of k times the size for an averaging loss such as ag_mse_loss; set it to FALSE for a summing loss, where the plain sum is already right.

Value

An optimizer environment with step(grads), zero_grad(), state() and load_state(state). state() returns a versioned list (moments as host double matrices, t, accum_count); load_state() checks the type, parameter count, names (when both sides have them) and shapes before writing anything, restores t and accum_count, and does NOT restore hyperparameters (it warns when they differ). Exact continuation holds on the same device and dtype; CPU <-> GPU in f32 continues within float tolerance; across f16 and f32 it cannot be exact. Only read saved states you trust (readRDS).

Examples

w <- ag_param(matrix(runif(4), 2, 2))
opt <- optimizer_sgd(list(w = w), lr = 0.01)

# accumulate 4 micro-batches per update
opt4 <- optimizer_sgd(list(w = w), lr = 0.01, accumulate_steps = 4L)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.