validate_rag_predictions: Validate RAG Emotion/Sentiment Predictions

In transforEmotion: Sentiment Analysis for Text, Image and Video using Transformer Models

Validate RAG Emotion/Sentiment Predictions

Description

Evaluates emotion/sentiment predictions from rag() or rag_sentemo() against ground truth labels using the same metrics pipeline as evaluate_emotions(). Supports table or JSON structured outputs.

Usage

validate_rag_predictions(
  rag_output,
  ground_truth,
  id_col = NULL,
  task = c("emotion", "sentiment"),
  labels_set = NULL,
  metrics = c("accuracy", "f1_macro", "confusion_matrix"),
  return_plot = FALSE
)

Arguments

rag_output	Output from rag() or rag_sentemo() with structured outputs (data.frame with columns like 'doc_id', 'label', 'confidence'; or JSON string with these fields). Global schema outputs with 'labels'/'confidences' are also handled by reducing to the top label.
ground_truth	Character vector of ground truth labels matching the number of predictions (or length of provided ids).
id_col	Optional identifier. If 'rag_output' is a data.frame and 'id_col' is a character scalar naming a column present in it, that column is used as the prediction id. Alternatively, 'id_col' can be a vector of ids (same length as 'ground_truth') used to align ground truth to the predictions by merge.
task	Task type: one of '"emotion"' or '"sentiment"' (used for metadata and optional label set enforcement).
labels_set	Optional character vector of allowed labels for validation. If provided, predictions will be lowercased and filtered to this set where possible.
metrics	Metrics to compute, forwarded to evaluate_emotions() (e.g., 'c("accuracy","f1_macro","confusion_matrix")').
return_plot	Logical; whether to include plotting helpers.

Value

A list of evaluation results in the same format as evaluate_emotions(), augmented with '$rag_metadata' summarizing RAG-specific context (documents, transformer, task).

Examples

## Not run: 
texts <- c(
  "I feel so happy and grateful today!",
  "This is frustrating and makes me angry.",
  "I'm not sure how I feel about this."
)

# Get predictions (structured per-document output)
rag_results <- rag_sentemo(
  texts,
  task = "emotion",
  output = "table",
  transformer = "Gemma3-1B"
)

# Ground truth labels
ground_truth <- c("joy", "anger", "neutral")

# Validate predictions
validation_results <- validate_rag_predictions(
  rag_output = rag_results,
  ground_truth = ground_truth,
  task = "emotion",
  metrics = c("accuracy", "f1_macro", "confusion_matrix"),
  return_plot = TRUE
)

## End(Not run)

transforEmotion documentation built on Jan. 8, 2026, 5:06 p.m.