ggml_predict: Get Predictions from a Trained Model

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Get Predictions from a Trained Model

Description

Runs forward pass on input data and returns prediction probabilities (or raw output values for regression). Unlike ggml_evaluate(), this does not require labels.

Usage

## S3 method for class 'ggml_functional_model'
ggml_predict(model, x, batch_size = 32L, matmul_precision = NULL, ...)

ggml_predict(model, ...)

## S3 method for class 'ggml_sequential_model'
ggml_predict(model, x, batch_size = 32L, matmul_precision = NULL, ...)

Arguments

model	A trained ggml_sequential_model
x	Input data (matrix or array)
batch_size	Batch size for inference
matmul_precision	"default" or "f32" for this call; NULL (the default) takes the model's predict_precision from ggml_compile. See the section on precision.
...	Additional arguments (ignored).

Value

Matrix of predictions with shape [N, output_units]

Precision

On a Vulkan device with fp16 support the matrix multiplies keep their tiles in f16 by default. Results then carry a relative error near 1e-3 that depends on the batch size (a single row takes an exact f32 path, larger batches the tiled one), so the same input can predict slightly differently in a batch of 1 and a batch of 64. "f32" marks every multiply GGML_PREC_F32 and selects f32 tiles: the result then matches the CPU to about 1e-6, at a cost in matmul throughput that depends on the device and driver. The CPU backend is exact either way.

Which setting wins: the matmul_precision argument of the call, then the model's predict_precision, then "default".

ggmlR documentation built on Oct. 23, 2026, 1:07 a.m.