call_llm_stream: Stream a chat completion token by token

In LLMR: Interface for Large Language Model APIs in R

Stream a chat completion token by token

Description

Like call_llm(), but the reply arrives incrementally: callback is invoked with each text chunk as it is generated, and the complete llmr_response is returned at the end. Streaming keeps long generations responsive and avoids HTTP timeouts on slow, lengthy completions.

Usage

call_llm_stream(
  config,
  messages,
  callback = function(chunk) cat(chunk),
  verbose = FALSE
)

Arguments

config	An llm_config for a generative model.
messages	Messages as in call_llm().
callback	Function called with each text chunk (a character scalar) as it arrives. The default prints chunks to the console with cat(). Reasoning deltas (when a provider streams them separately) are not passed to callback; they are collected into the result's thinking field.
verbose	Print the assembled response object at the end.

Details

Supported providers: all OpenAI-compatible chat APIs (openai, groq, together, deepseek, xai, alibaba, zhipu, moonshot, xiaomi, ollama), Anthropic, and Gemini. The request body is built by the same internals as call_llm(), so parameters and structured output behave identically; the request_modifier and req_builder hooks and the request timeout apply as well. Only the transport differs. The response_modifier hook is the one exception: it rewrites a full parsed response, which has no per-chunk meaning for a stream, so it is not applied here.

Value

An llmr_response assembled from the stream (invisibly). Token usage is filled when the provider reports it in the stream; otherwise it is NA.

See Also

call_llm(), chat_session()

Examples

## Not run: 
cfg <- llm_config("groq", "openai/gpt-oss-20b")
r <- call_llm_stream(cfg, "Tell a 100-word story about a lighthouse.")
tokens(r)

## End(Not run)

LLMR documentation built on July 11, 2026, 1:06 a.m.