Working with Embeddings

In foundryR: Data Frame Workflows for 'Microsoft Foundry'

# This vignette runs against recorded, credential-free API fixtures. When the
# fixtures are present (they are recorded once with data-raw/record-doc-outputs.R
# and committed under vignettes/embeddings/), every foundry_*() call below is
# executed and its real output is shown. When they are absent, the API chunks are
# not evaluated so the vignette still builds anywhere without Azure credentials.
fixture_dir <- "embeddings"
recording <- nzchar(Sys.getenv("FOUNDRY_RECORD_DOCS"))
have_fixtures <- dir.exists(fixture_dir) && length(list.files(fixture_dir)) > 0
run_api <- requireNamespace("httptest2", quietly = TRUE) &&
  (recording || have_fixtures)

# Attach foundryR before start_vignette(): httptest2 only sources the package's
# inst/httptest2/start-vignette.R (which sets replay placeholders) from attached
# packages.
library(foundryR)

if (run_api) {
  httptest2::start_vignette(fixture_dir)
}

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = run_api
)

What embeddings are for

Embeddings are numerical representations of text that capture semantic meaning. When you convert text to an embedding, you get a vector of numbers (often 1,536 or 3,072 dimensions depending on the model). Texts with similar meanings tend to have similar vectors.

Use embeddings when you need to:

• Find documents related to a query by meaning, not only keywords.
• Measure how similar two pieces of text are.
• Cluster open-ended responses into themes.
• Find near-duplicate responses or records.
• Feed text-derived numeric predictors into downstream models.

Unlike keyword matching, embeddings understand that "automobile" and "car" are semantically similar, even though they share no letters.

Generating embeddings with foundry_embed()

The examples below embed real sentences from Jane Austen's Pride and Prejudice, available in the janeaustenr package. Using a well-known public-domain text makes the output easy to reason about: the opening lines share vocabulary and sentiment, so their embeddings should sit close together.

library(foundryR)

austen_lines <- c(
  "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.",
  "However little known the feelings or views of such a man may be on his first entering a neighbourhood.",
  "Mr. Bennet was so odd a mixture of quick parts, sarcastic humour, reserve, and caprice."
)

embedding <- foundry_embed(austen_lines[1], model = "text-embedding-3-small")
embedding

The result is a tibble with:

• text: the original input text.
• embedding: a list-column holding the numeric vector.
• n_dims: the dimensionality of the embedding.

Embedding multiple texts

Pass a character vector to embed several texts in one call:

doc_embeddings <- foundry_embed(austen_lines, model = "text-embedding-3-small")
doc_embeddings

Controlling dimensions

The text-embedding-3-small and text-embedding-3-large models can return shorter vectors. Smaller dimensions mean faster similarity computations and less storage, with some trade-off in precision:

compact <- foundry_embed(
  austen_lines[1],
  model = "text-embedding-3-small",
  dimensions = 256
)
compact$n_dims

Computing similarity with foundry_similarity()

Cosine similarity measures how close two embeddings are, from -1 (opposite) to 1 (identical). foundry_similarity() computes every pairwise similarity in a tibble of embeddings. Here we contrast the Austen lines with two sentences from a different domain so the split is visible.

mixed <- c(
  "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.",
  "Mr. Bennet was so odd a mixture of quick parts, sarcastic humour, reserve, and caprice.",
  "The quarterly revenue report showed a sharp rise in cloud subscriptions.",
  "Analysts raised their earnings forecast after the strong cloud numbers."
)

similarities <- foundry_embed(mixed, model = "text-embedding-3-small") |>
  foundry_similarity()
similarities

Results are sorted by similarity. The two Austen lines pair together and the two finance lines pair together, while cross-domain pairs score lower. The plots in this vignette are shown when the suggested ggplot2 package is installed.

label_for <- function(x) {
  dplyr::case_when(
    startsWith(x, "It is a truth") ~ "Austen 1",
    startsWith(x, "Mr. Bennet") ~ "Austen 2",
    startsWith(x, "The quarterly") ~ "Finance 1",
    TRUE ~ "Finance 2"
  )
}

heatmap_data <- similarities |>
  dplyr::mutate(
    a = label_for(text_1),
    b = label_for(text_2)
  )

ggplot2::ggplot(heatmap_data, ggplot2::aes(x = b, y = a, fill = similarity)) +
  ggplot2::geom_tile(color = "white", linewidth = 0.6) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.2f", similarity)), size = 3.2) +
  ggplot2::scale_fill_gradient(low = "#E6F2FB", high = "#0078D4") +
  ggplot2::labs(
    title = "Cosine similarity separates the two domains",
    x = NULL, y = NULL, fill = "Similarity"
  ) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(panel.grid = ggplot2::element_blank(), legend.position = "bottom")

Use case: finding similar documents

A common application is ranking documents by relevance to a query. Embed the documents and the query, then sort by cosine similarity:

library(dplyr)

documents <- c(
  "How to install R packages using install.packages()",
  "Data visualization with ggplot2 in R",
  "Introduction to machine learning with Python",
  "Statistical hypothesis testing explained",
  "Building web applications with Shiny",
  "Deep learning with TensorFlow and Keras"
)

doc_embeddings <- foundry_embed(documents, model = "text-embedding-3-small")
query_embedding <- foundry_embed(
  "How do I create charts and graphs in R?",
  model = "text-embedding-3-small"
)

cosine <- function(a, b) sum(a * b) / (sqrt(sum(a^2)) * sqrt(sum(b^2)))
query_vec <- query_embedding$embedding[[1]]

doc_embeddings |>
  mutate(similarity = vapply(embedding, cosine, numeric(1), b = query_vec)) |>
  arrange(desc(similarity)) |>
  select(text, similarity) |>
  head(3)

Use case: clustering text

Embeddings work well as features for clustering. Here stats::kmeans() groups a mix of programming, food, and sports sentences without any labels:

texts <- c(
  "Python is great for machine learning",
  "R excels at statistical analysis",
  "JavaScript powers modern web applications",
  "Italian pasta with tomato sauce",
  "Sushi is a popular Japanese dish",
  "French croissants are flaky and buttery",
  "Soccer is the world's most popular sport",
  "Basketball requires speed and agility",
  "Tennis matches can last for hours"
)

cluster_embeddings <- foundry_embed(texts, model = "text-embedding-3-small")
embedding_matrix <- do.call(rbind, cluster_embeddings$embedding)

set.seed(42)
clusters <- kmeans(embedding_matrix, centers = 3, nstart = 10)

cluster_embeddings |>
  mutate(cluster = clusters$cluster) |>
  arrange(cluster) |>
  select(text, cluster)

pca <- prcomp(embedding_matrix, rank. = 2)
projection <- tibble::tibble(
  pc1 = pca$x[, 1],
  pc2 = pca$x[, 2],
  cluster = factor(clusters$cluster),
  label = substr(texts, 1, 18)
)

ggplot2::ggplot(projection, ggplot2::aes(pc1, pc2, color = cluster, label = label)) +
  ggplot2::geom_point(size = 3.2, alpha = 0.9) +
  ggplot2::geom_text(nudge_y = 0.15, size = 3, show.legend = FALSE) +
  ggplot2::scale_color_manual(values = c("#0078D4", "#107C10", "#5C2D91")) +
  ggplot2::labs(
    title = "A PCA projection makes the clusters visible",
    x = "PC 1", y = "PC 2", color = "Cluster"
  ) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(legend.position = "bottom", panel.grid.minor = ggplot2::element_blank())

The clusters recover the three topics from the raw text alone.

Tips for working with embeddings

Choosing a model

| Model | Dimensions | Notes | |-------|-----------|-------| | text-embedding-ada-002 | 1,536 | Previous generation, widely used | | text-embedding-3-small | 1,536 (configurable) | Newer, supports dimension reduction | | text-embedding-3-large | 3,072 (configurable) | Highest quality, more expensive |

For most use cases, text-embedding-3-small balances quality and cost.

Dimension trade-offs

Higher dimensions capture more nuance but need more storage, take longer to compare, and may not improve simple tasks. Consider reduced dimensions (256-512) for large-scale applications where speed matters more than precision.

Handling large collections

1. Batch processing: embed documents in batches to respect rate limits.
2. Caching: store embeddings in a database rather than regenerating them.
3. Approximate nearest neighbors: use libraries like RcppAnnoy for fast similarity search on large datasets.

# Defining this helper is local; calling it requires Azure credentials.
batch_embed <- function(texts, model, batch_size = 100) {
  n_batches <- ceiling(length(texts) / batch_size)
  results <- vector("list", n_batches)
  for (i in seq_len(n_batches)) {
    start_idx <- (i - 1) * batch_size + 1
    end_idx <- min(i * batch_size, length(texts))
    results[[i]] <- foundry_embed(texts[start_idx:end_idx], model = model)
    Sys.sleep(0.5)
  }
  dplyr::bind_rows(results)
}

if (run_api) {
  httptest2::end_vignette()
}

foundryR documentation built on Sept. 25, 2026, 1:10 a.m.