Audio Workflows with Microsoft Foundry

In foundryR: Data Frame Workflows for 'Microsoft Foundry'

# This vignette runs against recorded, credential-free API fixtures. The
# fixtures are recorded once by a maintainer with real Azure OpenAI credentials
# (data-raw/record-doc-outputs.R) and committed under vignettes/audio/. When the
# fixtures are present every foundry_*() call below executes and shows its real
# output; when they are absent the API chunks are not evaluated so the vignette
# still builds anywhere without credentials. No output on this page is fabricated.
fixture_dir <- "audio"
recording <- nzchar(Sys.getenv("FOUNDRY_RECORD_DOCS"))
have_fixtures <- dir.exists(fixture_dir) && length(list.files(fixture_dir)) > 0
run_api <- requireNamespace("httptest2", quietly = TRUE) &&
  (recording || have_fixtures)

# Attach foundryR before start_vignette(): httptest2 only sources the package's
# inst/httptest2/start-vignette.R (which sets replay placeholders) from attached
# packages.
library(foundryR)

if (run_api) {
  httptest2::start_vignette(fixture_dir)
}

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = run_api
)

# Embed a self-contained <audio> player holding the real bytes foundry_speak()
# produced, so readers can play the output (not just read its size). On replay
# the file holds the recorded audio, so this works offline with no credentials.
embed_audio <- function(path) {
  if (!requireNamespace("base64enc", quietly = TRUE)) {
    return(invisible(NULL))
  }
  uri <- paste0("data:audio/mpeg;base64,", base64enc::base64encode(path))
  knitr::asis_output(
    sprintf(
      '<audio controls src="%s">Your browser does not support audio playback.</audio>',
      uri
    )
  )
}

library(foundryR)

Audio is one of the most useful Foundry additions for researchers. You can transcribe interviews, lectures, field recordings, meetings, and focus groups, then keep the result in a tibble with segment-level timing for downstream coding or analysis.

Two ways to reach speech models

foundryR can use audio models in two places:

• Azure OpenAI deployments on your main resource -- whisper for transcription and translation, and a text-to-speech model such as gpt-4o-mini-tts for synthesis. These reuse your main endpoint and key and are what the examples below use. Classic whisper is exposed only on the deployment path, so pass api = "deployment".
• A dedicated Speech (LLM Speech) resource for MAI-Transcribe models. This chunk is illustrative and is not run:

# Only needed for MAI-Transcribe on a dedicated Speech resource:
foundry_set_speech_endpoint(Sys.getenv("AZURE_FOUNDRY_SPEECH_ENDPOINT"))
foundry_set_speech_key("your-speech-key")

A real, public-domain sample

The examples below use a short excerpt from John F. Kennedy's 1961 inaugural address ("And so, my fellow Americans..."). This clip ships with the package and is the de facto "hello, world" of open-source speech recognition, so the transcript is easy to check against a recording everyone knows.

sample_audio <- system.file("extdata/samples/jfk.wav", package = "foundryR")
basename(sample_audio)

Transcribe an audio file

foundry_transcribe() returns one row per file. The text column holds the transcript and the phrases list-column holds segment-level timing. We use the whisper deployment on the main resource; because classic whisper lives on the deployment path we pass api = "deployment", and response_format = "verbose_json" asks the service for per-segment timing.

transcript <- foundry_transcribe(
  sample_audio,
  service = "openai",
  model = "whisper",
  api = "deployment",
  response_format = "verbose_json"
)

transcript$text

The segment timing lives in the phrases list-column, one row per recognized segment. A short clip like this one is a single segment; longer recordings return many:

head(transcript$phrases[[1]])

Synthesize speech

foundry_speak() writes binary audio to disk and returns the file path and byte count -- handy for experiment stimuli, accessibility assets, and demos. Use your text-to-speech deployment name for model. The examples use temporary files and remove them after use; choose an explicit path in your own workflow for audio you want to keep.

speech_path <- tempfile(fileext = ".mp3")
speech <- foundry_speak(
  "Hello, world.",
  model = "gpt-4o-mini-tts",
  voice = "alloy",
  path = speech_path
)

speech[, c("bytes", "model", "voice", "format")]

Those bytes are the real audio the model returned. Play them here when the suggested base64enc package is installed:

embed_audio(speech$path)

Translate multilingual recordings

Use foundry_translate_audio() when you want an analysis corpus in a common language. To keep the example fully reproducible we first synthesize a short Spanish clip, then translate it to English with whisper -- both are real API calls.

spanish_path <- tempfile(fileext = ".mp3")
spanish_clip <- foundry_speak(
  "La reunion fue muy util.",
  model = "gpt-4o-mini-tts",
  voice = "alloy",
  path = spanish_path
)

Listen to the synthesized Spanish input:

embed_audio(spanish_clip$path)

Now translate it to English with the whisper deployment:

translation <- foundry_translate_audio(
  spanish_clip$path,
  service = "openai",
  model = "whisper",
  api = "deployment"
)

translation$text

Notes for researchers

• Inspect head(transcript$phrases[[1]]) before processing long recordings so you know the segment structure your coding scheme has to handle.
• Keep raw audio out of your project repository; store transcripts and IDs.
• Request response_format = "verbose_json" to get segment-level timing from whisper; the default format returns the transcript text only.

if (run_api) {
  unlink(c(speech_path, spanish_path))
  httptest2::end_vignette()
}

foundryR documentation built on Sept. 25, 2026, 1:10 a.m.