ggml_fit_opt_gen: Train from a generator, one batch at a time

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Train from a generator, one batch at a time

Description

Streaming counterpart of ggml_fit_opt. Instead of a ggml_opt_dataset holding every sample, batches are pulled from a generator function, so only the current batch is ever in memory. This is what makes three things possible that the dataset path cannot express: data larger than RAM, augmentation that must differ per epoch, and generated sources (RL rollouts, simulators) that have no fixed dataset at all.

Usage

ggml_fit_opt_gen(
  sched,
  ctx_compute,
  inputs,
  outputs,
  generator,
  steps_per_epoch = NULL,
  epochs = 1L,
  initial_epoch = 0L,
  validation_generator = NULL,
  validation_steps = NULL,
  loss_type = ggml_opt_loss_type_mse(),
  optimizer = ggml_opt_optimizer_type_adamw(),
  nbatch_logical = NULL,
  callbacks = list(),
  silent = FALSE,
  loss_mask_ne0 = NULL,
  weight_decay = 0
)

Arguments

sched	Backend scheduler
ctx_compute	Compute context holding the model graph
inputs	Input tensor of the built graph
outputs	Output tensor of the built graph
generator	Function of no arguments returning one batch, or NULL when exhausted. See the contract above.
steps_per_epoch	Number of batches that make up one epoch. Required when generator is infinite; when omitted, an epoch runs until the generator returns NULL.
epochs	Number of epochs to run.
initial_epoch	Epoch number to start counting from, as in Keras. This shifts the numbering seen by callbacks and learning-rate schedules only; it does not restore optimizer state. A fresh opt_ctx starts with zeroed Adam moments regardless, so this is a warm restart with the LR schedule resumed, not a full resume.
validation_generator	Optional generator used for validation after each epoch, following the same contract. Evaluated without a backward pass.
validation_steps	Number of validation batches per epoch. Required when validation_generator is infinite.
loss_type	Loss type constant
optimizer	Optimizer type constant
nbatch_logical	Logical batch size, for gradient accumulation
callbacks	List of callback lists, as for ggml_fit_opt
silent	Suppress per-epoch messages
loss_mask_ne0	Width of the per-output loss mask, or NULL
weight_decay	Decoupled weight decay in [0, 1], as for ggml_opt_init_for_fit; 0 turns it off.

Value

A history data frame, one row per epoch, as ggml_fit_opt returns.

The generator contract

generator is a function of no arguments. Each call returns one batch; returning NULL ends the epoch. The same contract covers both finite and infinite sources – an infinite generator simply never returns NULL, and steps_per_epoch then becomes required, because nothing else can say where an epoch ends.

A finite generator must restart itself. Training runs several epochs over the same source, so a generator that ends must reset its own state before returning NULL, leaving the next call to yield the first batch again. Nothing else can do it: a bare closure has no handle to reset from. An epoch that receives no batch at all stops training with a warning rather than looping over nothing.

batch_gen <- function(x, y, batch_size) {
  i <- 0L
  nb <- nrow(x) %/% batch_size
  function() {
    if (i >= nb) {
      i <<- 0L          # reset, so the next epoch starts over
      return(NULL)
    }
    idx <- (i * batch_size + 1L):((i + 1L) * batch_size)
    i <<- i + 1L
    list(x[idx, , drop = FALSE], y[idx, , drop = FALSE])
  }
}

A batch is list(inputs = list(...), labels = list(...)). For the common single-input case a plain list(x, y) is accepted as well. Each element is a numeric vector or array already in ggml layout, holding exactly one physical batch.

The generator is called from R, so it may do anything R can: read a shard from disk, augment the previous batch, or sample a replay buffer that a simulator is filling.

Gradient accumulation

When nbatch_logical exceeds the graph's physical batch size, the optimizer accumulates gradients over several generator batches and steps once per logical batch. The period is tracked inside opt_ctx; the generator is unaware of it and simply keeps producing physical batches.

See Also

ggml_fit_opt for the in-memory dataset path.

Other optimization: ggml_fit_opt(), ggml_fit_opt_multi(), ggml_opt_alloc(), ggml_opt_context_optimizer_type(), ggml_opt_dataset_data(), ggml_opt_dataset_free(), ggml_opt_dataset_get_batch(), ggml_opt_dataset_get_batch_head(), ggml_opt_dataset_init(), ggml_opt_dataset_labels(), ggml_opt_dataset_ndata(), ggml_opt_dataset_shuffle(), ggml_opt_dataset_weights(), ggml_opt_default_params(), ggml_opt_epoch(), ggml_opt_eval(), ggml_opt_fit(), ggml_opt_fit_multi(), ggml_opt_free(), ggml_opt_get_lr(), ggml_opt_grad_acc(), ggml_opt_init(), ggml_opt_init_for_fit(), ggml_opt_init_for_fit_multi(), ggml_opt_inputs(), ggml_opt_labels(), ggml_opt_loss(), ggml_opt_loss_i(), ggml_opt_loss_type_binary_cross_entropy(), ggml_opt_loss_type_cross_entropy(), ggml_opt_loss_type_huber(), ggml_opt_loss_type_mae(), ggml_opt_loss_type_mean(), ggml_opt_loss_type_mse(), ggml_opt_loss_type_sum(), ggml_opt_loss_type_weighted_mse(), ggml_opt_n_loss(), ggml_opt_ncorrect(), ggml_opt_optimizer_name(), ggml_opt_optimizer_type_adamw(), ggml_opt_optimizer_type_sgd(), ggml_opt_outputs(), ggml_opt_outputs_i(), ggml_opt_pred(), ggml_opt_prepare_alloc(), ggml_opt_reset(), ggml_opt_result_accuracy(), ggml_opt_result_free(), ggml_opt_result_init(), ggml_opt_result_loss(), ggml_opt_result_loss_i(), ggml_opt_result_n_loss(), ggml_opt_result_ndata(), ggml_opt_result_pred(), ggml_opt_result_reset(), ggml_opt_set_lr(), ggml_opt_static_graphs(), ggml_snapshot_free(), ggml_trainer()

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.