compute_confusion_matrices: Compute confusion matrices from multi-model annotations

In localLLM: Running Local LLMs with 'llama.cpp' Backend

Compute confusion matrices from multi-model annotations

Description

Compute confusion matrices from multi-model annotations

Usage

compute_confusion_matrices(
  annotations,
  gold = NULL,
  pairwise = TRUE,
  label_levels = NULL,
  sample_col = "sample_id",
  model_col = "model_id",
  label_col = "label",
  truth_col = "truth"
)

Arguments

annotations	Output from [explore()] or a compatible data frame with at least 'sample_id', 'model_id', and 'label' columns.
gold	Optional vector of gold labels. Overrides the 'truth' column when supplied.
pairwise	When 'TRUE', cross-model confusion tables are returned even if no gold labels exist.
label_levels	Optional factor levels to enforce a consistent ordering in the resulting tables.
sample_col, model_col, label_col, truth_col	Column names to use when 'annotations' is a custom data frame.

Value

A list with elements 'vs_gold' (named list of matrices, one per model) and 'pairwise' (list of pairwise confusion tables).

localLLM documentation built on Nov. 3, 2026, 1:09 a.m.