clip_grad_norm: Clip gradients by global L2 norm

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Clip gradients by global L2 norm

Description

Rescales all gradients in grads so that their global L2 norm does not exceed max_norm. Modifies the grads environment in-place and returns the pre-clip norm.

Usage

clip_grad_norm(params, grads, max_norm)

Arguments

params	Named list of ag_param tensors (same as passed to optimizer).
grads	Gradient environment returned by backward().
max_norm	Maximum allowed global L2 norm.

Details

Call this after backward() and before optimizer$step().

Value

Numeric: the global L2 norm before clipping (invisibly).

Examples

w  <- ag_param(matrix(runif(4), 2, 2))
x  <- ag_tensor(matrix(c(1, 1), 2, 1))
with_grad_tape({
  out  <- ag_matmul(w, x)
  loss <- ag_mse_loss(out, matrix(0, 2, 1))
})
grads <- backward(loss)
clip_grad_norm(list(w = w), grads, max_norm = 1.0)

ggmlR documentation built on July 14, 2026, 1:08 a.m.