ag_buffer: Device buffer for column-indexed reads

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Device buffer for column-indexed reads

Description

A matrix of rows x capacity kept on the compute device without a gradient: written in place by column ranges with ag_buffer_write() and read by column indices with ag_get_rows() inside ag_* computations, including ag_capture() recordings. Typical use is a replay buffer: one column per transition.

Usage

ag_buffer(rows, capacity, dtype = "f32")

ag_buffer_write(buf, data, col_offset)

ag_buffer_read(buf, col_offset = 0, k = buf$capacity - col_offset)

ag_buffer_free(buf)

Arguments

rows	Number of rows (values per column).
capacity	Number of columns.
dtype	Storage type. Only "f32" is supported.
buf	An ag_buffer.
data	Numeric matrix with rows rows (a vector is one column).
col_offset	0-based index of the first column written or read.
k	Number of columns to read.

Details

On the CPU the buffer is an R matrix and the same calls work on it. On the GPU the memory is allocated when the buffer is created (or first used on the device) and zero-filled; creation fails with an error when the buffer needs more than 85% of the free memory the device reports (driver estimates are approximate). A device switch (ag_device("cpu")) downloads the value to the host in 16 MB parts before the backend is released (the host copy is an R double matrix, about twice the buffer's device size), and the switch fails with an error if that download does; the next use on the GPU uploads it again, and recordings that read the buffer are then made anew.

Writes are synchronous: when ag_buffer_write() returns, the data is on the device, and any graph computed afterwards, including a captured replay, reads it. Graph nodes queued before the write are computed first, so they read the old value.

ag_buffer_write() writes columns col_offset to col_offset + ncol(data) - 1; the range must lie inside the buffer (a ring buffer wraps with two writes).

ag_buffer_read() returns columns col_offset to col_offset + k - 1 as a rows x k matrix; reading a large buffer in parts bounds the host memory used.

ag_buffer_free() releases the memory at once instead of at garbage collection; the buffer cannot be used afterwards.

Value

An object of class ag_buffer with fields rows, capacity and dtype.

See Also

ag_get_rows

Examples

buf <- ag_buffer(3, 10)
ag_buffer_write(buf, matrix(1:6, 3, 2), col_offset = 4)
ag_buffer_read(buf, 4, 2)
as.matrix(ag_get_rows(buf, c(5, 4, 5)))
ag_buffer_free(buf)

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.