transplantFormants: Transplant formants

In soundgen: Sound Synthesis and Acoustic Analysis

Transplant formants

Description

Takes the general spectral envelope of one sound (donor) and "transplants" it onto another sound (recipient). For biological sounds like speech or animal vocalizations, this has the effect of replacing the formants in the recipient sound while preserving the original intonation and (to some extent) voice quality. Note that freqWindow is a crucial parameter: too narrow, and noise between harmonics will be amplified, creasing artifacts; too wide, and formants may be missed. The default is to set freqWindow to the estimated median pitch, but this is time-consuming and error-prone, so set it to a reasonable value manually if possible. Also ensure that both sounds have the same sampling rate.

Usage

transplantFormants(
  donor,
  recipient,
  samplingRate = NULL,
  freqWindow = NULL,
  dynamicRange = 80,
  windowLength = 50,
  step = NULL,
  overlap = 90,
  wn = "gaussian",
  zp = 0
)

Arguments

donor	the sound that provides the formants (vector, Wave, or file) or the desired spectral filter (matrix) as returned by getSpectralEnvelope
recipient	the sound that receives the formants (vector, Wave, or file)
samplingRate	sampling rate of x (only needed if x is a numeric vector)
freqWindow	the width of smoothing window. Defaults to median pitch of the donor (or of the recipient if donor is a filter matrix)
dynamicRange	dynamic range, dB. All values more than one dynamicRange under maximum are treated as zero
windowLength	length of FFT window, ms
step	you can override overlap by specifying FFT step, ms (NB: because digital audio is sampled at discrete time intervals of 1/samplingRate, the actual step and thus the time stamps of STFT frames may be slightly different, eg 24.98866 instead of 25.0 ms)
overlap	overlap between successive FFT frames, %
wn	window type accepted by ftwindow, currently gaussian, hanning, hamming, bartlett, rectangular, blackman, flattop
zp	window length after zero padding, points

Details

Algorithm: makes spectrograms of both sounds, interpolates and smoothes the donor spectrogram, flattens the recipient spectrogram, multiplies the spectrograms, and transforms back into time domain with inverse STFT.

See Also

transplantEnv getSpectralEnvelope addFormants soundgen

Examples

## Not run: 
# Objective: take formants from the bleating of a sheep and apply them to a
# synthetic sound with any arbitrary duration, intonation, nonlinearities etc
data(sheep, package = 'seewave')  # import a recording from seewave
playme(sheep)
spectrogram(sheep, osc = TRUE)

recipient = soundgen(
  sylLen = 1200,
  pitch = c(100, 300, 250, 200),
  vibratoFreq = 9, vibratoDep = 1,
  addSilence = 180,
  samplingRate = [e-mail],  # same as donor
  invalidArgAction = 'ignore')  # force to keep the low samplingRate
playme(recipient, [e-mail])
spectrogram(recipient, [e-mail], osc = TRUE)

s1 = transplantFormants(
  donor = sheep,
  recipient = recipient,
  samplingRate = [e-mail])
playme(s1, [e-mail])
spectrogram(s1, [e-mail], osc = TRUE)

# The spectral envelope of s1 will be similar to sheep's on a frequency scale
# determined by freqWindow. Compare the spectra:
par(mfrow = c(1, 2))
seewave::meanspec(sheep, dB = 'max0', alim = c(-50, 20), main = 'Donor')
seewave::meanspec(s1, f = [e-mail], dB = 'max0',
                  alim = c(-50, 20), main = 'Processed recipient')
par(mfrow = c(1, 1))

# if needed, transplant amplitude envelopes as well:
s2 = transplantEnv(donor = sheep, samplingRateD = [e-mail],
                   recipient = s1, windowLength = 10)
playme(s2, [e-mail])
spectrogram(s2, [e-mail], osc = TRUE)

# Now we use human formants on sheep source: the sheep asks "why?"
s3 = transplantFormants(
  donor = getSpectralEnvelope(
            nr = 512, nc = 100,  # fairly arbitrary dimensions
            formants = 'uaaai',
            samplingRate = [e-mail]),
  recipient = sheep,
  samplingRate = [e-mail])
playme(s3, [e-mail])
spectrogram(s3, [e-mail], osc = TRUE)

## End(Not run)

soundgen documentation built on Sept. 29, 2023, 5:09 p.m.