ggml_vulkan_split_mul_mat: Tensor-parallel matrix multiply across Vulkan devices

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Tensor-parallel matrix multiply across Vulkan devices

Description

Computes Y = X %*% t(W) with the rows of the weight matrix W split across n_devices GPUs (true tensor parallelism), the activations X broadcast to every device, and the resulting column slices gathered back into a single matrix. This is Stage E3 of the Vulkan tensor-parallelism work: it exercises the row-split + per-device compute + cross-device gather path end to end on real hardware.

Usage

ggml_vulkan_split_mul_mat(
  W,
  X,
  n_devices = ggml_vulkan_device_count(),
  weights = NULL,
  device_ids = NULL,
  transport = c("host-staging", "opaque-fd", "device-group")
)

Arguments

W	Weight matrix, N x K (N output features, K input features). Its rows are split across the devices.
X	Activation matrix, M x K (M samples, K input features), broadcast to every device.
n_devices	Number of GPUs to split across (default: all available). Ignored if device_ids is given (its length is used instead).
weights	Optional numeric vector of length n_devices giving the relative share of W's rows per device (e.g. c(3, 1) for a 3:1 split). NULL (default) splits the rows evenly.
device_ids	Optional integer vector of physical GPU indices (0-based) to split across, e.g. c(2, 3) for the second replica of a TPxDP layout. NULL (default) uses devices 0..n_devices-1. When supplied, n_devices is taken from its length.
transport	Cross-device gather transport: "host-staging" (default, portable and correct on every driver), "opaque-fd" or "device-group" (see ggml_vulkan_p2p_selftest).

Details

Each device owns a contiguous band of W's rows (via the same row-split math as ggml_vulkan_split_row_ranges) and produces the matching band of Y's columns; the bands are gathered through the transport (host staging by default). This is a validation / correctness entry point, not a hot inference path — it stands up one backend per device per call and round-trips through host memory.

Value

The M x N result matrix, equal to X %*% t(W) up to the GPU's floating-point accumulation. Errors if the split compute fails; the C-level diagnostic is attached as attribute "report".

Examples

if (ggml_vulkan_available() && ggml_vulkan_device_count() >= 2) {
  W <- matrix(rnorm(8 * 4), nrow = 8)   # 8 output features x 4 inputs
  X <- matrix(rnorm(3 * 4), nrow = 3)   # 3 samples x 4 inputs
  Y <- ggml_vulkan_split_mul_mat(W, X, n_devices = 2)
  max(abs(Y - X %*% t(W)))              # ~1e-3 (GPU f16 accumulation)
}

ggmlR documentation built on July 14, 2026, 1:08 a.m.