ggml_backend_meta_device: Create a Meta Backend Device

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Create a Meta Backend Device

Description

Creates a "meta" device that wraps multiple "simple" backend devices for tensor parallelism. Each tensor is split across the wrapped devices according to the result of split_fn, which is called by ggml when weight buffers are allocated.

Usage

ggml_backend_meta_device(devs, split_fn, env = environment(split_fn))

Arguments

devs	A list of ggml_backend_dev_t external pointers.
split_fn	A function function(tensor_info, n_devs) returning the split state as described above.
env	An environment in which to evaluate split_fn; defaults to the function's enclosing environment.

Details

The split function is invoked with two arguments:

tensor_info

a named list with fields name (character), type (integer ggml_type enum), ne (numeric vector of dimensions), op (integer op enum), flags (integer).

n_devs

the number of simple devices wrapped by the meta backend.

It must return a named list with:

axis

integer; one of 0..3 to split along a tensor axis, 10 for MIRRORED (full copy on each device), 11 for PARTIAL (each device has a partial sum), or 98/99 for NONE/UNKNOWN.

ne

integer or numeric vector of length n_segments * n_devs giving the per-segment, per-device slice size along the split axis.

n_segments

integer; usually 1, larger for fused tensors like QKV.

If split_fn errors or returns an unparseable result, the meta backend silently falls back to MIRRORED for that tensor and stops calling the callback (sticky error). This is intentional: a misbehaving callback would otherwise spray errors for every tensor in the model.

Note: with a single device this is a degenerate (no-op) configuration — useful for testing but provides no parallelism benefit. The feature is experimental and the API may change.

Value

External pointer to the meta ggml_backend_dev_t.

See Also

Other backend: ggml_backend_buffer_clear(), ggml_backend_buffer_get_usage(), ggml_backend_buffer_is_host(), ggml_backend_buffer_is_multi_buffer(), ggml_backend_buffer_reset(), ggml_backend_buffer_set_usage(), ggml_backend_buffer_usage_any(), ggml_backend_buffer_usage_compute(), ggml_backend_buffer_usage_weights(), ggml_backend_dev_by_name(), ggml_backend_dev_by_type(), ggml_backend_dev_count(), ggml_backend_dev_description(), ggml_backend_dev_get(), ggml_backend_dev_get_props(), ggml_backend_dev_init(), ggml_backend_dev_memory(), ggml_backend_dev_name(), ggml_backend_dev_offload_op(), ggml_backend_dev_supports_buft(), ggml_backend_dev_supports_op(), ggml_backend_dev_type(), ggml_backend_device_register(), ggml_backend_device_type_accel(), ggml_backend_device_type_cpu(), ggml_backend_device_type_gpu(), ggml_backend_device_type_igpu(), ggml_backend_event_free(), ggml_backend_event_new(), ggml_backend_event_record(), ggml_backend_event_synchronize(), ggml_backend_event_wait(), ggml_backend_get_device(), ggml_backend_graph_compute_async(), ggml_backend_graph_plan_compute(), ggml_backend_graph_plan_create(), ggml_backend_graph_plan_free(), ggml_backend_init_best(), ggml_backend_init_by_name(), ggml_backend_init_by_type(), ggml_backend_load(), ggml_backend_load_all(), ggml_backend_multi_buffer_alloc_buffer(), ggml_backend_multi_buffer_set_usage(), ggml_backend_reg_by_name(), ggml_backend_reg_count(), ggml_backend_reg_dev_count(), ggml_backend_reg_dev_get(), ggml_backend_reg_get(), ggml_backend_reg_name(), ggml_backend_register(), ggml_backend_synchronize(), ggml_backend_tensor_copy_async(), ggml_backend_tensor_get_async(), ggml_backend_tensor_set_async(), ggml_backend_unload()

ggmlR documentation built on July 14, 2026, 1:08 a.m.