ggml_gated_linear_attn: Gated Linear Attention

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Gated Linear Attention

Description

Linear attention with a per-token gate (the GLA / RetNet family). Shares the RWKV recurrence machinery: keys, values and queries drive a running state that g gates, and scale multiplies the query.

Usage

ggml_gated_linear_attn(ctx, k, v, q, g, state, scale)

Arguments

ctx	GGML context
k	Keys, [S, H, n_tokens]
v	Values, [S, H, n_tokens]
q	Queries, [S, H, n_tokens]
g	Gate, [S, H, n_tokens]
state	Initial state, S * S * H * n_seqs elements
scale	Scalar applied to the queries (usually 1 / sqrt(S))

Value

A tensor [S * H, n_tokens + S * n_seqs]

Shapes

k, v, q and g are [S, H, n_tokens]; state holds S * S * H * n_seqs elements. The result is [S * H, n_tokens + S * n_seqs], outputs then final state – see ggml_rwkv_output.

n_tokens counts the tokens of all sequences together, so it must be a multiple of n_seqs.

Backend support

CPU only. Unlike the RWKV and SSM ops, GGML_OP_GATED_LINEAR_ATTN has no Vulkan shader, so a graph containing it falls back to the CPU for this node. It has no backward pass either, so it is inference-only.

See Also

ggml_rwkv_wkv7, ggml_rwkv_output

Other state-space: ggml_gated_linear_attn_back(), ggml_rwkv_output(), ggml_rwkv_wkv6(), ggml_rwkv_wkv6_back(), ggml_rwkv_wkv7(), ggml_rwkv_wkv7_back(), ggml_ssm_conv(), ggml_ssm_conv_back(), ggml_ssm_scan(), ggml_ssm_scan_back(), ggml_ssm_scan_output()

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.