ggml_layer_sequence_pooling: Pool a Sequence Down to One Vector

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Pool a Sequence Down to One Vector

Description

Collapses the sequence axis, turning c(seq_len, d_model) into d_model: the step that lets a classification or regression head sit on top of an encoder. ggml_layer_flatten() also produces a flat vector, but keeps every position separately (seq_len * d_model features), so its width – and the head above it – depends on the sequence length. Pooling does not.

Usage

ggml_layer_sequence_pooling(model, mode = c("mean", "first"), name = NULL)

Arguments

model	A ggml_sequential_model or a ggml_tensor_node carrying a sequence of shape c(seq_len, d_model).
mode	"mean" (default) or "first".
name	Optional character name for the layer.

Details

mode = "mean" averages over the positions; every position contributes equally, which is the usual choice for an encoder without a dedicated summary token. mode = "first" takes position 1 and ignores the rest – the CLS-token convention, where attention is expected to have gathered what matters into that position.

Value

The model with the layer appended, or a new ggml_tensor_node of shape d_model.

See Also

ggml_layer_flatten, ggml_layer_attention

Examples

# An encoder with a regression head: the head's width is d_model, whatever
# the sequence length turns out to be.
x <- ggml_input(shape = c(10L, 32L))
h <- x |> ggml_layer_attention(d_model = 32L, n_heads = 4L)
h <- h |> ggml_layer_sequence_pooling()
y <- h |> ggml_layer_dense(1L)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.