ggml_rwkv_wkv7: RWKV-7 WKV Recurrence

In ggmlR: 'GGML' Tensor Operations for Machine Learning

RWKV-7 WKV Recurrence

Description

The RWKV-7 ("Goose") recurrence. Where RWKV-6 decays the state by a vector, RWKV-7 applies a rank-1 update built from a and b alongside the decay w, giving the state a richer transition.

Usage

ggml_rwkv_wkv7(ctx, r, w, k, v, a, b, state)

Arguments

ctx	GGML context
r	Receptance, [S, H, n_tokens]
w	Decay, [S, H, n_tokens]
k	Keys, [S, H, n_tokens]
v	Values, [S, H, n_tokens]
a	Removal (state update term), [S, H, n_tokens]
b	Addition (state update term), [S, H, n_tokens]
state	Initial state, S * S * H * n_seqs elements

Value

A tensor [S * H, n_tokens + S * n_seqs]

Shapes

r, w, k, v, a and b are [S, H, n_tokens]; state holds S * S * H * n_seqs elements. All must be contiguous. The result is [S * H, n_tokens + S * n_seqs], outputs then final state – see ggml_rwkv_output.

n_tokens counts the tokens of all sequences together, so it must be a multiple of n_seqs.

Inference only

GGML_OP_RWKV_WKV7 has no backward pass in ggml.

See Also

ggml_rwkv_wkv6, ggml_rwkv_output

Other state-space: ggml_gated_linear_attn(), ggml_gated_linear_attn_back(), ggml_rwkv_output(), ggml_rwkv_wkv6(), ggml_rwkv_wkv6_back(), ggml_rwkv_wkv7_back(), ggml_ssm_conv(), ggml_ssm_conv_back(), ggml_ssm_scan(), ggml_ssm_scan_back(), ggml_ssm_scan_output()

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.