rank_records: Rank a corpus with an LLM ensemble

In screenllm: LLM-Assisted Title/Abstract Screening for Systematic Reviews

Rank a corpus with an LLM ensemble

Description

Sends every record to every model in the ensemble at every replicate, aggregates the resulting scores per record, and returns the corpus with the aggregated score and rank attached. Progress is displayed with cli. All intermediate scores are cached to disk so an interrupted run can be resumed by calling rank_records() again with the same cache_dir.

Usage

rank_records(
  records,
  criteria,
  ensemble = default_ensemble(),
  cache_dir = getOption("screenllm.cache_dir"),
  verbose = getOption("screenllm.verbose", TRUE),
  max_workers = getOption("screenllm.max_workers", 1L),
  on_score = NULL
)

Arguments

records	A tibble of records (produced by read_records()).
criteria	A screenllm_criteria object.
ensemble	A screenllm_ensemble object. Defaults to default_ensemble().
cache_dir	Directory to write per-call cache files. If NULL, a temporary directory is used and the results are not persisted between R sessions.
verbose	Show progress messages and bars.
max_workers	Not yet used; reserved for future concurrency.
on_score	Optional callback invoked once per (record, model, replicate) tuple after each score is available (whether fresh or restored from cache). Signature: ⁠function(id, model, replicate, score, explanation, error, index, total)⁠. Used by the Shiny app's ranking module to stream partial scores into the UI as they land; safe to leave NULL.

Value

The input tibble with added columns: universal_best_score, rank, per_model_scores (list-column), justifications (list-column of per-criterion rationales as returned by the LLM).

Examples

# A no-Ollama, in-R demo using the mock backend.
records <- data.frame(
  id = c("a", "b"),
  title = c("Coral reef restoration outcomes",
            "Deep-sea mining impacts on benthic fauna"),
  abstract = c("We monitored transplanted corals for 12 months...",
               "This modelling study estimates long-term sediment plumes...")
)
criteria <- define_criteria(
  scope = "Field-based coral reef restoration and performance",
  inclusions = c("Study is field-based.", "Study describes an intervention.")
)
ens <- default_ensemble(backend = backend_mock())
ranked <- rank_records(records, criteria, ensemble = ens, verbose = FALSE)
ranked[, c("id", "universal_best_score", "rank")]

screenllm documentation built on Sept. 24, 2026, 5:11 p.m.