quick_llama: Quick LLaMA Inference

In localLLM: Running Local LLMs with 'llama.cpp' Backend

Quick LLaMA Inference

Description

A high-level convenience function that provides one-line LLM inference. Automatically handles model downloading, loading, and text generation with optional chat template formatting and system prompts for instruction-tuned models.

Usage

quick_llama(
  prompt,
  model_path = .get_default_model(),
  n_threads = NULL,
  n_gpu_layers = "auto",
  n_ctx = 2048L,
  verbosity = 1L,
  max_tokens = 100L,
  top_k = 40L,
  top_p = 1,
  temperature = 0,
  repeat_last_n = 0L,
  penalty_repeat = 1,
  system_prompt = "You are a helpful assistant.",
  auto_format = TRUE,
  chat_template = NULL,
  seed = 1234L,
  progress = interactive(),
  clean = FALSE,
  hash = TRUE,
  ...
)

Arguments

prompt	Character string or vector of prompts to process
model_path	Model URL or path (default: Llama 3.2 3B Instruct Q5_K_M)
n_threads	Number of threads (default: auto-detect)
n_gpu_layers	Number of GPU layers (default: auto-detect)
n_ctx	Context size (default: 2048)
verbosity	Backend logging verbosity (default: 1L). Higher values show more detail: 0 prints only errors, 1 adds warnings, 2 includes informational messages, and 3 enables the most verbose debug output.
max_tokens	Maximum tokens to generate (default: 100)
top_k	Top-k sampling (default: 40). Limits vocabulary to k most likely tokens
top_p	Top-p sampling (default: 1.0). Set to 0.9 for nucleus sampling
temperature	Sampling temperature (default: 0.0). Higher values increase creativity
repeat_last_n	Number of recent tokens to consider for repetition penalty (default: 0). Set to 0 to disable. Must be non-negative
penalty_repeat	Repetition penalty strength (default: 1.0). Set to 1.0 to disable
system_prompt	System prompt to add to conversation (default: "You are a helpful assistant.")
auto_format	Whether to automatically apply chat template formatting (default: TRUE)
chat_template	Custom chat template to use (default: NULL uses model's built-in template)
seed	Random seed for reproducibility (default: 1234)
progress	Show a console progress bar during parallel generation. Defaults to interactive(). Has no effect for single-prompt runs.
clean	Whether to strip chat-template control tokens from the generated output. Defaults to FALSE.
hash	When 'TRUE' (default), compute SHA-256 hashes for the prompts fed into the backend and the corresponding outputs. Hashes are attached via the '"hashes"' attribute for later inspection.
...	Additional parameters passed to generate() or generate_parallel()

Value

Character string (single prompt) or named list (multiple prompts)

See Also

model_load, generate, generate_parallel, install_localLLM

Examples

## Not run: 
# Simple usage with default settings (deterministic)
response <- quick_llama("Hello, how are you?")

# Raw text generation without chat template
raw_response <- quick_llama("Complete this: The capital of France is",
                           auto_format = FALSE)

# Custom system prompt
code_response <- quick_llama(
  "Write a Python hello world program",
  system_prompt = "You are a Python programming expert.")

# Creative writing with higher temperature
creative_response <- quick_llama("Tell me a story",
                                 temperature = 0.8,
                                 max_tokens = 200)

# Prevent repetition
no_repeat <- quick_llama("Explain AI",
                        repeat_last_n = 64,
                        penalty_repeat = 1.1)

# Multiple prompts (parallel processing)
responses <- quick_llama(c("Summarize AI", "Explain quantum computing"),
                        max_tokens = 150)

## End(Not run)

localLLM documentation built on Nov. 3, 2026, 1:09 a.m.