ggml_vulkan_split_buffer_type: Create a Vulkan tensor-split buffer type

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Create a Vulkan tensor-split buffer type

Description

Builds (or fetches from a cache) a ggml buffer type that row-splits a weight tensor across n_devices GPUs for tensor parallelism. This is the factory the graph allocator talks to (Stage E4): a weight placed in this buffer type has its rows distributed across the devices, each holding a contiguous band (see ggml_vulkan_split_row_ranges). Non-split tensors fall back to main_device.

Usage

ggml_vulkan_split_buffer_type(
  n_devices = ggml_vulkan_device_count(),
  main_device = 0L,
  weights = NULL,
  device_ids = NULL,
  transport = c("host-staging", "opaque-fd", "device-group"),
  probe = NULL
)

Arguments

n_devices	Number of GPUs to split across (default: all available). Ignored if device_ids is given (its length is used instead).
main_device	Device index (0-based) holding non-split fallbacks (default 0).
weights	Optional numeric vector of length n_devices giving the relative row share per device (e.g. c(3, 1)). NULL (default) splits evenly.
device_ids	Optional integer vector of physical GPU indices (0-based) the split slots map onto, e.g. c(2, 3). NULL (default) uses 0..n_devices-1. When supplied, n_devices is its length.
transport	Cross-device gather transport for the eventual compute: "host-staging" (default), "opaque-fd" or "device-group".
probe	A numeric vector c(N, K) describing a probe weight tensor (N rows, K columns) whose split allocation size to report, or NULL (default) to skip. Useful for inspecting how much total VRAM a given weight would occupy across the split.

Details

The buffer type is cached in C and lives for the rest of the session; the returned external pointer has no finalizer and must not be freed.

Value

A named list: ptr (external pointer to the cached buffer type), name (its config-derived name) and alloc_size (sum of the padded per-device slice sizes for probe, in bytes; 0 if no probe).

Examples

if (ggml_vulkan_available() && ggml_vulkan_device_count() >= 2) {
  bt <- ggml_vulkan_split_buffer_type(n_devices = 2, probe = c(2048, 64))
  bt$name
  bt$alloc_size   # total bytes for a 2048x64 f32 weight across 2 devices
}

ggmlR documentation built on July 14, 2026, 1:08 a.m.