ggml_freeze_bn_stats: Fix or Release Batch-Norm Statistics

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Fix or Release Batch-Norm Statistics

Description

ggml_freeze_bn_stats() fixes the running mean and variance of batch-norm layers: they then normalize with those stored statistics during training as well as at prediction, and ggml_fit no longer recalibrates them. gamma and beta still train unless the layer is also frozen with ggml_freeze_weights – the usual fine-tuning setting when batches are small. ggml_unfreeze_bn_stats() returns the layers to batch statistics in training and recalibration after a fit, which is the default.

Usage

ggml_freeze_bn_stats(
  model,
  from = 1L,
  to = length(model$layers),
  layer_names = NULL,
  prefix = NULL,
  ...
)

ggml_unfreeze_bn_stats(
  model,
  from = 1L,
  to = length(model$layers),
  layer_names = NULL,
  prefix = NULL,
  ...
)

Arguments

model	A model object (ggml_sequential_model or ggml_functional_model)
from	Integer index of the first layer to freeze (default: 1)
to	Integer index of the last layer to freeze (default: last layer)
layer_names	Character vector of layer names to freeze (overrides from/to)
prefix	Character vector: freeze every layer whose name starts with one of these – a whole transformer block, say: prefix = "block1_" reaches block1_norm1, block1_attn, block1_norm2, block1_ff1 and block1_ff2 – not block10_*. Given with layer_names, the selection is the union of the two; every name and every prefix must match at least one layer, or the call stops.
...	Additional arguments passed to methods

Details

Freezing the weights alone does not fix the statistics: a frozen batch-norm still normalizes each training batch by its own mean and variance, and its running statistics are still recalibrated after ggml_fit().

The fixed state is saved by ggml_save_model. Only sequential models carry running statistics; the functional batch_norm is a per-sample RMS normalization (see ggml_layer_batch_norm), already independent of the batch, and is refused with a message.

Value

The model with the selected batch-norm layers changed. Selected layers of other types are skipped; a selection with no batch-norm layer at all is an error.

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.