llm_batch_submit: Submit a batch job to a provider's batch API

In LLMR: Interface for Large Language Model APIs in R

Submit a batch job to a provider's batch API

Description

Provider batch APIs run large jobs asynchronously at a reduced price (typically half) in exchange for delayed delivery (minutes up to 24 hours). llm_batch_submit() packages one request per element of messages and submits the job; llm_batch_status() polls it; llm_batch_fetch() retrieves results as a tidy tibble aligned with the inputs.

Usage

llm_batch_submit(config, messages, state_path = NULL)

Arguments

config	An llm_config for a generative model.
messages	An unnamed character vector (each element becomes one request's user message); a named character vector like c(system = ..., user = ...) (a single multi-role request); or a list with one element per request, where each element is any messages form accepted by call_llm(). Multimodal file parts are not supported in batch jobs.
state_path	Optional file path; when given, the job object is also saved there as RDS (and the path is remembered for convenience).

Details

Supported providers: "openai" and "groq" (Files + Batches protocol), "anthropic" (Message Batches), and "gemini" (batchGenerateContent with inline requests; developer API, not Vertex). All request-shaping features of llm_config() apply: sampling parameters, structured output, tools, and hooks shape each request exactly as a live call_llm() would.

The returned job object contains no secrets (the config stores an environment-variable reference, not the key), so it can be saved to disk, shared, and fetched later or from another machine with the same environment variables set. Pass state_path to save it automatically.

Value

An llmr_batch_job object.

Batching, chunking, and row packing

"Batch" appears in three distinct senses in LLMR, and they are easy to keep apart once you note which path each belongs to. (1) The asynchronous provider Batch API (llm_batch_submit() and friends) defers a whole job for later delivery at a reduced price; it is the only deferred path here. (2) In the embedding path, get_batched_embeddings()'s batch_size sets how many texts go in one synchronous embedding request, bounded by the provider's per-call limit. (3) In the generative path, the .rows_per_prompt argument here packs several data rows into one generative prompt and parses them back into rows. Senses (2) and (3) are synchronous: batch_size and .rows_per_prompt control how work is grouped into requests, not the per-token rate, so the synchronous helpers do not themselves apply a provider discount. Pricing is a separate axis: several providers bill batched embeddings at a reduced rate through a dedicated async/batch tier (the embedding analogue of (1)), so embeddings are not categorically full-price; that discount is a property of the endpoint, not of batch_size. Row packing (3) can still lower total tokens by amortizing shared prompt overhead across rows, again without changing the rate.

See Also

llm_batch_status(), llm_batch_fetch(), llm_batch_cancel(); and, for the two synchronous senses of "batch", get_batched_embeddings() (embedding chunking) and llm_mutate() (.rows_per_prompt row packing).

Examples

## Not run: 
cfg <- llm_config("groq", "openai/gpt-oss-20b", temperature = 0)
job <- llm_batch_submit(cfg, c("2+2?", "Capital of Chile?"),
                        state_path = "my_batch.rds")
llm_batch_status(job)
# ... later, possibly in a new session:
res <- llm_batch_fetch("my_batch.rds")

## End(Not run)

LLMR documentation built on July 11, 2026, 1:06 a.m.