ggml_layer_positional_embedding: Add a Learned Positional Embedding

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Add a Learned Positional Embedding

Description

Adds a learned vector to each position of a sequence, so that attention – which is order-blind on its own – can tell one position from another. The table is [d_model, seq_len], one row per position, added to the input and broadcast over the batch.

Usage

ggml_layer_positional_embedding(model, name = NULL, trainable = TRUE)

Arguments

model	A ggml_sequential_model or a ggml_tensor_node carrying a sequence of shape c(seq_len, d_model).
name	Optional character name for the layer.
trainable	Logical; whether the table is updated during training.

Details

Unlike ggml_layer_embedding there are no indices to look up: the position IS the place in the sequence, so this layer takes the sequence itself and returns it with the positional term added. The shape is unchanged.

Value

The model with the layer appended, or a new ggml_tensor_node of the same shape as the input.

See Also

ggml_layer_attention, ggml_layer_embedding

Examples

# Without this the encoder cannot distinguish "ab" from "ba".
x <- ggml_input(shape = c(10L, 32L))
h <- x |> ggml_layer_positional_embedding()
h <- h |> ggml_layer_attention(d_model = 32L, n_heads = 4L)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.