ag_graph_mode: Graph mode for ag_* operations on the GPU

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Graph mode for ag_* operations on the GPU

Description

With graph mode on, a GPU ag_* operation that has a device-resident path does not run at once: it adds its node to a queue and returns a pending value. The queue runs as ONE ggml graph when a value is read on the host (as.matrix(), a printed loss, an action index) or when backward() needs it – the graph backward then joins the same graph. Each separate launch costs on the order of a millisecond on the device, so a small network (an RL policy, a few layers) is dominated by launches, and fusing them is where the speed comes from.

Usage

ag_graph_mode(on, check = NULL)

Arguments

on	TRUE or FALSE. Missing: return the current setting.
check	TRUE/FALSE to switch the per-operation check on or off; NULL (default) leaves it as it is.

Details

Results are the same as without it. Operations that have no resident path yet still run one by one; they first run whatever is queued, so the order of computation is always the data order.

When the queue runs (synchronisation points):

• reading a value that is still queued: as.matrix(), printing, a loss value, ag_argmax();

• an operation without a resident path (it runs the queue first);

• writing a parameter: an optimizer step, ag_load_state_dict();

• the device optimizer step (it never joins the queue);

• turning graph mode off.

backward() does NOT force it: the graph backward appends to the queue, so forward and backward run as one graph at the next point above.

Off by default; turn it on around the code that wants it, restoring the previous value on every exit – ag_local_mode does that. The one thing it can break is code that orders graphs BY HAND rather than through data – reading a buffer in one graph and overwriting it in a later one with nothing linking the two. The package's own optimizer step does this and is protected; code of your own that does it must run with graph mode off. On the CPU it has no effect.

Errors. A failure inside a queued operation surfaces where the queue runs, not where the operation was called. The error names the queued operations; check = TRUE (or GGMLR_AG_GRAPH_CHECK=1) runs each operation as soon as it is queued, so the error appears at its own call – for debugging, since it gives up the fusion.

Value

The current setting when on is missing; otherwise the previous setting, invisibly. Turning it off runs anything still queued.

See Also

ag_local_mode, ag_matmul_precision

Examples

old <- ag_graph_mode(TRUE)
ag_graph_mode()
ag_graph_mode(old)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.