dragon_metrics: Task metrics for generated replies

In dragonfarm: Fine-Tune Small Language Models with LoRA from R

Task metrics for generated replies

Description

Deterministic checks that need no judge model. Each metric is a function of three character vectors, generated, reference, and prompt, and returns one number per row (0 or 1 for pass/fail metrics). Pass names from this list, or your own functions, to dragon_evaluate().

Usage

dragon_metrics()

dragon_metric_regex(pattern, ignore_case = TRUE)

Arguments

pattern	A regular expression the reply must match.
ignore_case	Case-insensitive match.

Details

• exact: normalized exact match (case, whitespace, and trailing punctuation ignored).

• contains: the normalized reference appears inside the reply.

• token_f1: token overlap F1 between reply and reference, the SQuAD style partial-credit score.

• json_valid: the reply parses as JSON (a fenced code block is unwrapped first).

• numeric: the last number in the reply equals the last number in the reference.

• length_ratio: characters in the reply divided by characters in the reference. Useful for spotting rambling or truncation.

dragon_metric_regex() builds a metric that passes when the reply matches a pattern, for format checks such as "starts with a ticket id".

Value

dragon_metrics(): a named list of metric functions.

dragon_metric_regex(): a metric function.

Examples

m <- dragon_metrics()
m$exact("Paris.", "paris", "Capital of France?")
m$token_f1("the cat sat on the mat", "a cat sat on a mat", "")
m$json_valid('```json\n{"a": 1}\n```', "", "")

dragonfarm documentation built on Oct. 12, 2026, 5:07 p.m.