ggml_gelu_back: GELU Backward (Graph)

In ggmlR: 'GGML' Tensor Operations for Machine Learning

GELU Backward (Graph)

Description

Gradient of the GELU activation: dx = dy * dgelu(x). Wired into the autodiff engine, so ggml_compile(activation = "gelu") trains – this is a ggmlR extension, upstream ggml has no GELU backward and aborts instead.

Usage

ggml_gelu_back(ctx, a, b)

Arguments

ctx	GGML context
a	Gradient tensor from upstream (dy)
b	Forward input tensor (x)

Details

The derivative is that of the tanh approximation ggml_gelu computes, g(x) = 0.5x(1 + tanh(u)) with u = c*x*(1 + a*x^2). The quick and erf variants (ggml_gelu_quick, ggml_gelu_erf) are different functions with different derivatives and remain without a backward rule.

Value

Gradient tensor for the input

See Also

ggml_gelu, ggml_silu_back

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.