foundry_eval_create: Create an evaluation

In foundryR: Data Frame Workflows for 'Microsoft Foundry'

Create an evaluation

Description

Create an evaluation group that pairs a data-source configuration with one or more graders (testing_criteria). Evaluations are run against data with foundry_eval_run_create().

Usage

foundry_eval_create(
  name = NULL,
  data_source_config,
  testing_criteria,
  metadata = NULL,
  api_key = NULL,
  token = NULL,
  endpoint = NULL,
  api_version = NULL
)

Arguments

name	Character. Optional evaluation name.
data_source_config	List. A configuration from foundry_eval_data_config().
testing_criteria	List. A grader from ⁠foundry_grader_*()⁠, or a list of graders.
metadata	List. Optional metadata attached to the evaluation.
api_key	Character. Optional API key. Falls back to configured auth.
token	Character. Optional bearer token. Falls back to configured auth.
endpoint	Character. Optional endpoint override.
api_version	Character. Optional api-version query value. The Foundry v1 evals surface is path-versioned, so this is usually left NULL.

Value

A one-row tibble describing the created evaluation.

Examples

## Not run: 
# Requires a configured Azure endpoint and credentials with evals API access.
foundry_eval_create(
  name = "qa-accuracy",
  data_source_config = foundry_eval_data_config(
    type = "custom",
    item_schema = list(
      type = "object",
      properties = list(answer = list(type = "string")),
      required = list("answer")
    ),
    include_sample_schema = TRUE
  ),
  testing_criteria = foundry_grader_string_check(
    name = "exact",
    input = "{{sample.output_text}}",
    reference = "{{item.answer}}",
    operation = "eq"
  )
)

## End(Not run)

foundryR documentation built on Sept. 25, 2026, 1:10 a.m.