spectrogram: Spectrogram

In soundgen: Sound Synthesis and Acoustic Analysis

Spectrogram

Description

Produces the spectrogram of a sound using short-time Fourier transform with added routines for reassignment, multi-resolution spectrograms, noise reduction, smoothing in time and frequency domains, manual control of contrast and brightness, plotting the oscillogram on a dB scale, etc. Gallery of examples: https://cogsci.se/soundgen/spectrograms.html.

Usage

spectrogram(
  x,
  samplingRate = NULL,
  scale = NULL,
  from = NULL,
  to = NULL,
  dynamicRange = 80,
  windowLength = 50,
  step = NULL,
  overlap = 50,
  specType = c("spectrum", "reassigned", "derivative"),
  logSpec = if (specType == "derivative") FALSE else TRUE,
  rasterize = FALSE,
  wn = "gaussian",
  zp = NULL,
  smoothFreq = 0,
  smoothTime = 0,
  qTime = 0,
  percentNoise = 10,
  noiseReduction = 0,
  output = c("original", "processed", "complex", "all", "none"),
  specManual = NULL,
  reportEvery = NULL,
  cores = 1,
  plot = TRUE,
  savePlots = FALSE,
  embed = FALSE,
  osc = c("linear", "dB", "none"),
  heights = c(3, 1),
  ylim = NULL,
  yScale = c("linear", "log", "bark", "mel", "ERB"),
  contrast = 0,
  brightness = 0,
  blur = 0,
  maxPoints = c(1e+05, 5e+05),
  padWithSilence = TRUE,
  colorTheme = "bw",
  col = NULL,
  extraContour = NULL,
  xlab = NULL,
  ylab = NULL,
  xaxp = NULL,
  mar = c(5.1, 4.1, 4.1, 2),
  main = NULL,
  grid = NULL,
  width = 900,
  height = 500,
  units = "px",
  res = NA,
  ...
)

Arguments

x	path to a folder, one or more wav or mp3 files c('file1.wav', 'file2.mp3'), Wave object, numeric vector, or a list of Wave objects or numeric vectors
samplingRate	sampling rate of x (only needed if x is a numeric vector)
scale	maximum possible amplitude of input, used to normalize the input vector (only needed if x is a numeric vector)
from, to	if NULL (default), analyzes the whole sound, otherwise from...to (s)
dynamicRange	regions under -dynamicRange dB are treated as silent
windowLength	length of FFT window, ms (multiple values in a vector produce a multi-resolution spectrogram, in which case step should have the same length)
step	step between successive windows, ms; if provided, overrides overlap; because digital audio is sampled at discrete time intervals of 1/samplingRate, the actual step and thus the time stamps of STFT frames may be slightly different - e.g., 24.98866 instead of 25.0 ms
overlap	overlap between successive windows, %
specType	plot the original FFT ('spectrum'), reassigned spectrogram ('reassigned'), or spectral derivatives ('derivative')
logSpec	if TRUE, log-transforms the spectrogram; no effect if specType = 'derivative' because then the spectrogram is always log-transformed
rasterize	(only applies if specType = 'reassigned') if TRUE, the reassigned spectrogram is plotted after rasterizing it: that is, showing density per time-frequency bins with the same resolution as an ordinary spectrogram
wn	window type accepted by winFun: character string or function
zp	window length after zero padding, samples (a vector of the same length as windowLength). No padding is performed if zp < analysis window length in samples. If NULL, the signal is padded to a good number for speeding up the FFT. For ex., if windowLength = 10 and samplingRate = 22050, each frame is 22050 * 0.01 = 220.5 samples long, so we can set zp = 512 or just default to nextn(22050 * 0.01) by setting zp = NULL. NB: do not use zero-padding (set zp = 0) if you need to invert the spectrogram afterwards with istft_simple or invertSpectrogram
smoothFreq, smoothTime	length of the window for median smoothing in frequency (Hz) and time domains (ms), respectively
qTime	the quantile to be subtracted for each frequency bin. For ex., if qTime = 0.5, the median of each frequency bin (over the entire sound duration) will be calculated and subtracted from each frame (see examples)
noiseReduction, percentNoise	how much noise to remove (non-negative number, recommended 0 to 2). 0 = no noise reduction, 2 = strong noise reduction: spectrum - (noiseReduction * noiseSpectrum), where noiseSpectrum is the average spectrum of frames with entropy exceeding the quantile set by percentNoise. percentNoise can range from 0 to 100 (default 10), and it gives the percentage of frames classified as noise
output	specifies what to return: unmodified spectrogram ('original'), denoised and/or smoothed spectrogram ('processed'), unmodified spectrogram with the imaginary part ('complex'), all of the above ('all'), or nothing ('none')
specManual	manually calculated spectrogram-like representation in the same format as the output of spectrogram(): rows = frequency in kHz, columns = time in ms
reportEvery	when processing multiple inputs, report estimated time left every reportEvery iterations (NULL = default, NA = don't report); see reportTime
cores	number of cores for parallel processing
plot	if TRUE, produces a plot of the results
savePlots	if TRUE, creates a subdirectory in the input directory (if input is a file or folder) or in the working directory (if input is a vector etc), named after the function (eg "spectrogram/"). All plots and audio files (if any) are saved in this new directory. If there are multiple inputs, an html notebook is also created for easy viewing and listening
embed	if TRUE and savePlots is set and there are multiple inputs, all saved images and audio (if any) are embedded in the exported html notebook for easy sharing; if FALSE, the html file links to separate images and audio files (but separate files are still saved). NB: for this to work, package "base64enc" must be installed
osc	"none" = no oscillogram; "linear" = on the original scale; "dB" = in decibels
heights	a vector of length two specifying the relative height of the spectrogram and the oscillogram (including time axes labels)
ylim	frequency range to plot, kHz (defaults to 0 to Nyquist frequency). NB: still in kHz, even if yScale = bark, mel, or ERB
yScale	scale of the frequency axis: 'linear' = linear, 'log' = logarithmic (musical), 'bark' = bark, 'mel' = mel, 'ERB' = Equivalent Rectangular Bandwidths
contrast	controls the sharpness or contrast of the image: <0 = decrease contrast, 0 = no change, >0 increase contrast. Recommended range approximately (-1, 1). The spectrogram is raised to the power of exp(3 * contrast)
brightness	makes the image lighter or darker, range [-1, 1] (default 0 = no change); for colorTheme = "bw", <0 = darker, >0 = lighter, range [-1, 1]. Values are remapped through a smooth sigmoid transfer curve that preserves the full color palette. To lighten or darken the palette itself, change the colors
blur	apply a Gaussian filter to blur or sharpen the image, two numbers: frequency (Hz), time (ms). A single number is interpreted as frequency, and a square filter is applied. NA / NULL / 0 means no blurring in that dimension. Negative numbers lead to unsharp masking with the default amount of 0.5 instead of blurring - see gaussianSmooth2D
maxPoints	the maximum number of "pixels" in the oscillogram (if any) and spectrogram; good for quickly plotting long audio files; defaults to c(1e5, 5e5); does not affect reassigned spectrograms
padWithSilence	if TRUE, pads the sound with half a window length of silence to resolve the edges properly (not needed if your sound is already padded with some silence)
colorTheme	black and white ('bw'), as in seewave package ('seewave'), matlab-type palette ('matlab'), or any palette from palette such as 'heat.colors', 'cm.colors', etc
col	actual colors, eg rev(rainbow(100)) - see ?hcl.colors for colors in base R (overrides colorTheme)
extraContour	a vector of arbitrary length scaled in Hz (regardless of yScale, but nonlinear yScale also warps the contour) that will be plotted over the spectrogram (eg pitch contour); can also be a list with extra graphical parameters such as lwd, col, warp (FALSE = plot as is, TRUE = warp to conform to nonlinear yScale), etc. (see examples)
xlab, ylab, main, mar, xaxp	graphical parameters for plotting
grid	if numeric, adds n = grid dotted lines per kHz and the same number of lines along the time axis
width, height, units, res	graphical parameters for saving plots passed to png
...	other graphical parameters

Details

Many soundgen functions call spectrogram, and you can pass along most of its graphical parameters from functions like soundgen, analyze, etc. However, in some cases this will not work (eg for "units") or may produce unexpected results. If in doubt, omit extra graphical parameters or save your sound first, then call spectrogram() explicitly. Reassigned spectrograms are not affected by noise reduction or blurring.

Value

Depends on the output argument. All matrices have frequency in rows (rownames in kHz) and time in columns (colnames in ms):

'none'

Nothing: only produces and/or saves the plot.

'original'

Unprocessed spectral magnitudes (not power!), with frequency bins from 0 up to and including Nyquist. If specType = 'derivative', the matrix contains spectral derivatives; if specType = 'reassigned', it contains either the rasterized reassigned spectrogram (matrix) or the unrasterized reassigned spectrogram - a dataframe giving time (ms), frequency (KHz), and magnitude of each bin).

'processed'

The same matrix as 'original' after denoising, time- / frequency-smoothing, blurring, and applying contrast and brightness.

'complex'

The complete complex STFT spectrogram with both positive and negative frequencies (rows beyond Nyquist correspond to negative frequencies, as in stft_simple). This is the raw Fourier transform prior to reassignment and any post-processing, suitable for custom frequency-domain manipulation followed by inverse STFT with istft_simple (NB: use zp = 0 if you intend to invert the spectrogram).

'all'

A list with all of the above.

For multi-resolution spectrograms (windowLength with several values), the plot combines all resolutions, but the returned matrices correspond to the last value of windowLength.

See Also

osc modulationSpectrum ssm

Examples

# Gallery of examples: https://cogsci.se/soundgen/spectrograms.html

# synthesize a sound 500 ms long, with gradually increasing hissing noise
sound = soundgen(sylLen = 500, temperature = 0.001, noise = list(
  time = c(0, 650), value = c(-40, 0)), formantsNoise = list(
  f1 = list(freq = 5000, width = 10000)))
# playme(sound, samplingRate = 16000)

# basic spectrogram
spectrogram(sound, samplingRate = 16000, yScale = 'bark')

# add bells and whistles
spectrogram(sound, samplingRate = 16000,
  windowLength = c(5, 40),  # multi-resolution
  osc = 'dB',  # plot oscillogram in dB
  heights = c(2, 1),  # spectro/osc height ratio
  contrast = .25,  # increase contrast
  brightness = -.3,  # reduce brightness
  # pick color theme - see ?hcl.colors
  # colorTheme = 'heat.colors',
  # ...or just specify the actual colors
  col = colorRampPalette(c('white', 'yellow', 'red'))(50),
  cex.lab = .75, cex.axis = .75,  # text size and other base graphics pars
  grid = 5,  # lines per kHz; to customize, add manually with graphics::grid()
  ylim = c(0, 5),  # always in kHz
  main = 'My spectrogram' # title
  # + axis labels, etc
)
## Not run: 
# save spectrograms of all sounds in a folder
spectrogram('~/Downloads/temp', savePlots = TRUE, cores = 2)

# change dynamic range
spectrogram(sound, samplingRate = 16000, dynamicRange = 40)
spectrogram(sound, samplingRate = 16000, dynamicRange = 120)

# remove the oscillogram
spectrogram(sound, samplingRate = 16000, osc = 'none')

# frequencies on a logarithmic (musical) scale (mel/bark/ERB also available)
spectrogram(sound, samplingRate = 16000,
            yScale = 'log', ylim = c(.05, 8))

# broad-band instead of narrow-band
spectrogram(sound, samplingRate = 16000, windowLength = 5)

# reassigned spectrograms can be plotted without rasterizing, as a
# scatterplot instead of a contour plot
s = soundgen(sylLen = 500, pitch = c(100, 1100, 120, 1200, 90, 900, 110, 700),
  samplingRate = 22050, formants = NULL, lipRad = 0, rolloff = -20)
spectrogram(s, 22050, windowLength = 5, step = 1, yScale = 'bark')
sp = spectrogram(s, 22050, specType = 'reassigned', windowLength = 5,
  step = 1, yScale = 'bark', output = 'original')
# ...or it can be rasterized, but that sacrifices frequency resolution:
sp2 = spectrogram(s, 22050, specType = 'reassigned', rasterize = TRUE,
                 windowLength = 5, step = 1, yScale = 'bark')

# an unrasterized reassigned spectrogram is just a dataframe of points
sp$z1 = soundgen:::zeroOne(log(sp$magn))
plot(sp$time, sp$freq, col = rgb(sp$z1, sp$z1, 1 - sp$z1, 1),
  pch = 16, cex = 0.5, ylim = c(0, 2))

# multi-resolution spectrograms
spectrogram(s, 22050, windowLength = c(1, 10, 20, 50), yScale = 'bark')
# (works well in combination with de-blurring)
spectrogram(s, 22050, windowLength = c(1, 10, 20, 50), yScale = 'bark',
  blur = c(-50, -50))
spectrogram(s, 22050, windowLength = 1:10, yScale = 'bark',
  specType = 'reassigned', dynamicRange = 50)
spectrogram(s, 22050, windowLength = 1:10, yScale = 'bark',
  specType = 'reassigned', dynamicRange = 50, rasterize = TRUE)

# Different combinations of specType, mono/multiresolution, and rasterization
spectrogram(s, 22050, windowLength = 5)
spectrogram(s, 22050, windowLength = c(5, 10))

spectrogram(s, 22050, windowLength = 5, specType = 'reassigned',
  rasterize = FALSE)
spectrogram(s, 22050, windowLength = c(5, 10), specType = 'reassigned',
  rasterize = FALSE)

spectrogram(s, 22050, windowLength = 5, specType = 'reassigned',
  rasterize = TRUE)
spectrogram(s, 22050, windowLength = c(5, 10), specType = 'reassigned',
  rasterize = TRUE)

# focus only on values in the upper 5% for each frequency bin
spectrogram(sound, samplingRate = 16000, qTime = 0.95)

# detect 10% of the noisiest frames based on entropy and remove the pattern
# found in those frames (in this case, breathing)
spectrogram(sound, samplingRate = 16000, noiseReduction = 0.2)

# increase contrast, reduce brightness
spectrogram(sound, samplingRate = 16000, contrast = .7, brightness = -.7)

# increase brightness (drops quiet bins with the same color palette)
spectrogram(sound, samplingRate = 16000, brightness = .5)

# another approach is to just make the palette lighter:
spectrogram(sound, samplingRate = 16000, col = gray.colors(30, 1, .5))

# median smoothing in time and/or frequency
spectrogram(sound, samplingRate = 16000, smoothFreq = 250)
spectrogram(sound, samplingRate = 16000, smoothTime = 100)
spectrogram(sound, samplingRate = 16000, smoothFreq = 250, smoothTime = 100)

# Gaussian filter to blur or sharpen the image in time and/or frequency
spectrogram(sound, samplingRate = 16000, blur = c(100, 50))
# TIP: when unblurring, set the first (frequency) parameter to the
# frequency resolution of interest, eg ~500-1000 Hz for human formants
spectrogram(sound, samplingRate = 16000, windowLength = 10, blur = c(-500, 50))

# specify location of tick marks etc - see ?par() for base graphics
spectrogram(sound, samplingRate = 16000,
            ylim = c(0, 3), yaxp = c(0, 3, 5), xaxp = c(0, .8, 10))

# Plot long audio files with reduced resolution
data(speechEx, package = 'soundgen')
sp = spectrogram(speechEx, windowLength = c(5, 25), yScale = 'ERB',
  maxPoints = c(1e4, 1e4),  # limit the number of pixels in osc/spec
  output = 'original')
nrow(sp) * ncol(sp) / 5e3  # spec downsampled (~54 times fewer pixels)

# Plot some arbitrary contour over the spectrogram (simply calling lines()
# will not work if the osc is shown because the plot layout is modified)
s = soundgen(sylLen = 1500, pitch = c(250, 350, 320, 220),
  jitterDep = c(0, 0, 3, 2, 0, 0))
an = analyze(s, 16000, plot = TRUE, extraContour = 'dom')
spectrogram(s, 16000, extraContour = an$detailed$dom,
  ylim = c(0, 2), yScale = 'bark')
spectrogram(s, 16000, extraContour = list(x = an$detailed$dom, col = 'green'),
  ylim = c(0, 2), yScale = 'bark')
# or simply add whatever you like to a spectrogram with points(), lines(),
# etc., (but only works without an oscillogram):
spectrogram(s, 16000, ylim = c(0, 2), yScale = 'bark', osc = 'none')
points(an$detailed$time/1000,  # time in s
       HzToOther(an$detailed$dom, 'bark'),  # values in barks
       lwd = 2, col = 'green', lty = 2)  # any graphic pars

# For values that are not in Hz, normalize any way you like. NB: if yScale !=
# 'linear', the extra contour is by default warped to the same scale b/c it
# is assumed to be in Hz. Specify "warp = FALSE" to avoid this
spectrogram(s, 16000, yScale = 'ERB', ylim = c(0, 5), extraContour = list(
  x = an$detailed$loudness / max(an$detailed$loudness, na.rm = TRUE) * 5000,
  # because ylim[2] = 5000 Hz
  type = 'b', pch = 5, lwd = 2, lty = 2, col = 'blue', warp = FALSE))
# compare:
spectrogram(s, 16000, yScale = 'ERB', ylim = c(0, 5), extraContour = list(
  x = an$detailed$loudness / max(an$detailed$loudness, na.rm = TRUE) * 5000,
  # because ylim[2] = 5000 Hz
  type = 'b', pch = 5, lwd = 2, lty = 2, col = 'blue'))

# Plot a spectrogram-like matrix paired with an osc
ms = modulationSpectrum(s, 16000, msType = '1D', amRes = 10)
spectrogram(s, 16000, specManual = ms$detailed$modulation_spectrogram,
  colorTheme = 'matlab', ylab = 'Modulation frequency, kHz',
  contrast = .25, blur = c(10, 10), yScale = 'log')

## End(Not run)

soundgen documentation built on Sept. 20, 2026, 5:07 p.m.