dragon_evaluate: Evaluate a finished run

In dragonfarm: Fine-Tune Small Language Models with LoRA from R

Evaluate a finished run

Description

Training already evaluates on the held-out rows and writes the result into the run directory. This function reads that result, or recomputes it with the saved adapter when recompute = TRUE or nothing was saved.

Usage

dragon_evaluate(run, n_samples = 10, recompute = FALSE, metrics = NULL)

Arguments

run	A dragon_run or run directory.
n_samples	Number of held-out prompts to generate replies for. Inf generates for every held-out row.
recompute	Reload the model and evaluate again.
metrics	Task metrics to compute on the generated replies: TRUE for every built-in metric, a character vector of names from dragon_metrics(), or a named list of functions. Metrics need a reply for every held-out row, so replies are generated for all of them when fewer are on disk. Results are saved to metrics.json in the run and show up in dragon_compare().

Value

A list with eval_loss, perplexity, eval_tokens, and a samples data frame with columns prompt, reference, generated. Preference runs report pref_accuracy (how often the model scores the chosen reply above the rejected one), reward_margin, and eval_pairs instead of perplexity, and their samples also carry rejected.

dragonfarm documentation built on Oct. 12, 2026, 5:07 p.m.