step_foundry_embed: Foundry Embedding Recipe Step

In foundryR: Data Frame Workflows for 'Microsoft Foundry'

Foundry Embedding Recipe Step

Description

Create text embeddings using an Azure AI Foundry model as part of a tidymodels recipe. This step converts text columns into embedding features for downstream modeling tasks such as classification, regression, or clustering.

Usage

step_foundry_embed(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  model = NULL,
  dimensions = NULL,
  prefix = "emb_",
  keep_original = FALSE,
  cache = c("none", "disk"),
  cache_dir = NULL,
  columns = NULL,
  skip = FALSE,
  id = NULL
)

## S3 method for class 'step_foundry_embed'
tidy(x, ...)

Arguments

recipe	A recipe object. The step will be added to the sequence of operations for this recipe.
...	Not used
role	Character. Role for the new embedding variables. Default: "predictor".
trained	Logical. Internal use only. Indicates whether the step has been trained.
model	Character. The deployment name of an Azure AI Foundry embedding model (e.g., "text-embedding-ada-002", "text-embedding-3-small"). If NULL, defaults to the AZURE_FOUNDRY_EMBED_MODEL environment variable.
dimensions	Integer or NULL. The number of dimensions for the output embeddings. Only supported by some models (e.g., text-embedding-3-*). If NULL, uses the model's default dimensionality.
prefix	Character. Prefix for the new embedding column names. Default: "emb_". Columns will be named ⁠{prefix}{original_col}_{1}⁠, ⁠{prefix}{original_col}_{2}⁠, etc.
keep_original	Logical. Should the original text column(s) be retained? Default: FALSE.
cache	Character. Embedding cache mode. "none" (default) always calls the API; "disk" caches each text's embedding on disk (keyed on the text, model, and dimensions) so cross-validation folds and repeated bakes reuse embeddings instead of re-calling the API.
cache_dir	Character. Directory for the disk cache. Defaults to a package-specific directory inside tempdir(), lasting only for the current R session. Supply a directory explicitly to persist embeddings across sessions. Clear it with foundry_cache_clear().
columns	Character vector. Internal use only. Stores column names after training.
skip	Logical. Should the step be skipped when the recipe is baked? While all operations are baked when recipes::prep() is run, some operations may not be applicable to new data (e.g., processing the outcome variable). Default: FALSE.
id	Character. Unique identifier for this step. Automatically generated if not provided.
x	A step_foundry_embed object

Details

This step uses foundry_embed() to generate embeddings for each text column specified. During the bake phase, each text value is sent to the Azure AI Foundry API, and the resulting embedding vector is expanded into multiple numeric columns.

Column naming

For a text column named "description" with 1536-dimensional embeddings and the default prefix "emb_", the output columns will be named: emb_description_1, emb_description_2, ..., emb_description_1536.

Handling failures

If an embedding request fails for a particular row (e.g., due to API errors), the corresponding embedding columns will be filled with NA values for that row.

Performance considerations

Embedding generation requires API calls for each unique text value. For large datasets or resampling, consider:

• Setting cache = "disk" so repeated bakes and cross-validation folds reuse embeddings instead of re-calling the API

• Using skip = TRUE during cross-validation to avoid redundant API calls

• Using batch processing strategies for very large datasets

Value

An updated recipe object with the new step appended to the sequence of existing steps.

A tibble with columns: terms, model, dimensions, id

See Also

foundry_embed() for the underlying embedding function, recipes::recipe() for creating recipes, recipes::prep() and recipes::bake() for processing recipes.

Examples

# Loading the optional modeling packages can take more than five seconds.
if (requireNamespace("recipes", quietly = TRUE)) {
df <- data.frame(
  text = c("Hello world", "Machine learning is great", "R is awesome"),
  category = c("greeting", "tech", "tech")
)

rec <- recipes::recipe(~ text, data = df) |>
  step_foundry_embed(text, model = "text-embedding-ada-002")
rec
}


## Not run: 
# Requires recipes, an Azure embedding deployment, endpoint, and credentials.
df <- data.frame(text = c("Hello world", "Machine learning is great"))
rec <- recipes::recipe(~ text, data = df) |>
  step_foundry_embed(
    text, model = "text-embedding-3-small", dimensions = 256,
    cache = "disk", cache_dir = file.path(tempdir(), "example-embeddings")
  )
prepped <- recipes::prep(rec, training = df)
baked <- recipes::bake(prepped, new_data = df)
foundry_cache_clear(file.path(tempdir(), "example-embeddings"))

## End(Not run)

foundryR documentation built on Sept. 25, 2026, 1:10 a.m.