foundry_eval_data_config: Define an evaluation data-source configuration

In foundryR: Data Frame Workflows for 'Microsoft Foundry'

Define an evaluation data-source configuration

Description

Describe the shape of the data an evaluation expects. type = "custom" declares an item schema you populate per run; type = "logs" sources rows from stored completions matching a metadata filter.

Usage

foundry_eval_data_config(
  type = c("custom", "logs"),
  item_schema = NULL,
  include_sample_schema = FALSE,
  metadata = NULL
)

Arguments

type	Character. Either "custom" or "logs".
item_schema	List. For type = "custom", a JSON Schema (as an R list) describing each row.
include_sample_schema	Logical. For type = "custom", whether the eval should expect a populated sample namespace (generated responses). Defaults to FALSE.
metadata	List. For type = "logs", the stored-completions metadata filter.

Value

A named list describing a data_source_config, for use in foundry_eval_create().

Examples

foundry_eval_data_config(
  type = "custom",
  item_schema = list(
    type = "object",
    properties = list(
      question = list(type = "string"),
      answer = list(type = "string")
    ),
    required = list("question", "answer")
  ),
  include_sample_schema = TRUE
)

foundryR documentation built on Sept. 25, 2026, 1:10 a.m.