generate: Generate Text Using Language Model Context

In localLLM: Running Local LLMs with 'llama.cpp' Backend

Generate Text Using Language Model Context

Description

Generates text using a loaded language model context with automatic tokenization. Simply provide a text prompt and the model will handle tokenization internally. This function now has a unified API with generate_parallel.

Usage

generate(
  context,
  prompt,
  max_tokens = 100L,
  top_k = 40L,
  top_p = 1,
  temperature = 0,
  repeat_last_n = 0L,
  penalty_repeat = 1,
  seed = 1234L,
  clean = FALSE,
  hash = TRUE,
  verbosity = 0L
)

Arguments

context	A context object created with context_create
prompt	Character string containing the input text prompt
max_tokens	Maximum number of tokens to generate (default: 100). Higher values produce longer responses
top_k	Top-k sampling parameter (default: 40). Limits vocabulary to k most likely tokens. Use 0 to disable
top_p	Top-p (nucleus) sampling (default: 1.0). Probability threshold for token selection.
temperature	Sampling temperature (default: 0.0). Set to 0 for greedy decoding. Higher values increase creativity
repeat_last_n	Number of recent tokens to consider for repetition penalty (default: 0). Set to 0 to disable. Must be non-negative
penalty_repeat	Repetition penalty strength (default: 1.0). Values >1 discourage repetition. Set to 1.0 to disable
seed	Random seed for reproducible generation (default: 1234). Use positive integers for deterministic output
clean	If TRUE, strip common chat-template control tokens from the generated text (default: FALSE).
hash	When 'TRUE' (default), computes SHA-256 hashes for the provided prompt and the resulting output. Hashes are attached via the '"hashes"' attribute for later inspection.
verbosity	Control backend logging during generation (default: 0L). Larger numbers print more detail: 0 shows only errors, 1 adds warnings, 2 prints informational messages, and 3 enables the most verbose debug output. Negative values fully suppress backend output. Defaults to quiet (0) because generate() is typically called inside loops where per-call backend logs would be noisy. This differs from model_load and context_create (default 1L), which run once per session and benefit from warnings being visible. Raise to 2L or 3L when debugging llama.cpp internals.

Value

Character string containing the generated text

See Also

quick_llama, generate_parallel, context_create

Examples

## Not run: 
# Load model and create context
model <- model_load("path/to/model.gguf")
ctx <- context_create(model, n_ctx = 2048)

response <- generate(ctx, "Hello, how are you?", max_tokens = 50)

# Creative writing with higher temperature
story <- generate(ctx, "Once upon a time",
                  max_tokens = 200, temperature = 0.8)

# Prevent repetition
no_repeat <- generate(ctx, "Tell me about AI",
                     repeat_last_n = 64,
                     penalty_repeat = 1.1)

# Clean output (remove special tokens)
clean_output <- generate(ctx, "Explain quantum physics", clean = TRUE)

## End(Not run)

localLLM documentation built on Nov. 3, 2026, 1:09 a.m.