ggml_layer_rms_norm: Add RMS Normalization Layer

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Add RMS Normalization Layer

Description

Normalizes each sample by the root mean square of its features, then scales by gamma and shifts by beta (both learnable). Unlike ggml_layer_batch_norm it does not subtract the mean and keeps no running statistics, so it behaves identically in training and inference and does not couple the samples in a batch. This is the normalization used by transformer blocks.

Usage

ggml_layer_rms_norm(model, eps = 1e-05, name = NULL, trainable = TRUE)

Arguments

model	A ggml_sequential_model or a ggml_tensor_node (functional API).
eps	Small constant for numerical stability (default 1e-5)
name	Optional character name for the layer.
trainable	Logical; whether the layer weights are updated during training.

Value

The model with the layer appended, or a new ggml_tensor_node in the functional API.

See Also

ggml_layer_batch_norm

Examples

# Functional API: the normalization of a transformer block.
x <- ggml_input(shape = c(10L, 32L))
h <- x |> ggml_layer_rms_norm()
h <- h |> ggml_layer_attention(d_model = 32L, n_heads = 4L)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.