In tuomaseerola/onsetsynch: Analysis and Visualisation of Synchronisation of Music Onset Data

knitr::opts_chunk$set(echo = TRUE)

Onsetsync demo as Rmd notebook

A Rmarkdown notebook with code examples in R for onsetsync by Tuomas Eerola, Music and Science Lab at Durham University and Music and Science Lab.

To run the code in your RStudio, download the Rmd file and open it in your Studio. Here's the link: https://github.com/tuomaseerola/onsetsync/onsetsync.Rmd. It assumes that you have devtools package installed to be able to install packages from github.

File onsetsync.Rmd | Version 9/11/2022 | Back to Index

Load libraries

library(ggplot2,quietly = TRUE)
library(tidyverse,quietly = TRUE)
require(devtools)
devtools::install_github("tuomaseerola/onsetsync")
library(onsetsync,quietly = TRUE)
ver <- packageVersion('onsetsync')
print(ver)

Load data (internal)

Here we take a Cuban Son performance titled Palo Santo from IEMP dataset [@Clayton2020emr] at https://osf.io/sfxa2/:

CSS_Song2 <- onsetsync::CSS_IEMP[[2]]   # Read one song from internal data
CSS_Song2 <- dplyr::select(CSS_Song2,Label.SD,SD,Clave,Bass,Guitar,Tres,
                           Cycle,Isochronous.SD.Time) # Select some columns
print(head(CSS_Song2))

Load Data -- From OSF (optional)

require(httr)                           # Additional library needed
CSS_Song2_Onset <- get_OSF_csv('8a347') # Get onsets
print(head(CSS_Song2_Onset[,1:8,]))

Summarise Raw Data

tab1 <- summarise_onsets(df = CSS_Song2, 
                         instr = c('Clave','Bass','Guitar','Tres'))
print(knitr::kable(tab1,format = "markdown", digits = 1,
     caption = 'Onsets counts (N) and mean IOIs in ms.'))

Visualisation

Visualise Onset Structures

fig1 <- plot_by_beat(df = CSS_Song2, 
                     instr = c('Bass','Clave','Guitar','Tres'), 
                     beat = 'SD', 
                     virtual = 'Isochronous.SD.Time',
                     pcols = 2)
print(fig1)

Visualise Deviations from Isochrony

fig2 <- plot_by_beat(df = CSS_Song2,
                     instr = c('Bass','Tres'),
                     beat = 'SD',
                     virtual = 'Isochronous.SD.Time', 
                     pcols=1,
                     griddeviations = TRUE)
print(fig2)

Visualise Across Sub-beats

fig2 <- plot_by_beat(df = CSS_Song2,
                     instr = c('Tres'),
                     beat = 'SD',
                     virtual = 'Isochronous.SD.Time', 
                     pcols=1,
                     griddeviations = TRUE, box = TRUE)
print(fig2)

Synchrony

Classic Measures of Synchrony

See definitions in classic literature (Rasch, Wing, Clayton).

d <- sync_sample_paired(CSS_Song2,'Clave','Bass',N=0,beat = 'SD')
s <- summarise_sync(d)
S <- data.frame(s); St<-t(S); colnames(St) <- 'ms'
knitr::kable(St,digits = 1,caption = 'Classic asynchrony measures.')

Synchrony Between Instruments

Number of shared onsets may vary across instruments/beats. Comparisons with sampling and bootstrapping.

set.seed(1201) # set random seed
N <- 100       # Let's select random 100 onsets from both
Boot <- 1      # No multiple bootstraps
d1 <- sync_sample_paired(CSS_Song2,'Clave','Bass', N, Boot,'SD',TRUE)
print(paste('Mean asynchrony of',round(mean(d1$asynch*1000),1),
    'ms & standard deviation of',round(sd(d1$asynch*1000),1),'ms'))

Synchrony Between Instruments (plot)

inst <- c('Clave','Bass','Guitar','Tres') # Define instruments 
dn <- sync_execute_pairs(CSS_Song2,inst,N,10,'SD')
print(plot_by_pair(dn))  # plot

Descriptive Statistics

table3 <- data.frame(summarise_sync_by_pair(dn))
print(knitr::kable(table3,
                   digits = 1,
                   caption = 'Descriptives, t-test and p-value.'))

Synchrony with other Variables

Is there a link between beat durations and synchrony?

CSS_Song2 <- CSS_Song2 %>% 
  group_by(Cycle) %>%
  mutate(Duration = max(Isochronous.SD.Time) - min(Isochronous.SD.Time))
CSS_Song2 <- ungroup(CSS_Song2) # drop grouping structure
d1 <- sync_sample_paired(df = CSS_Song2,
                         INSTR1 = 'Clave', INSTR2 = 'Bass',
                         N = 200, BNum = 1, beat = 'Duration')
fig6 <- plot_by_variable(d1,meta = 'Clave-Bass Synchrony',xlab = 'Cycle duration (in s)')
print(fig6)

Synchrony with other Variables

Are there specific trends in synchrony across the performance?

d2 <- sync_sample_paired(CSS_Song2,'Tres','Bass',N=0,beat='Duration')
d3 <- sync_sample_paired(CSS_Song2,'Tres','Bass',N=0,beat='Isochronous.SD.Time')
tmp <- data.frame(asynch=d2$asynch*1000,Duration=d2$beatL,Time=d3$beatL)
fig9 <- plot_by_var_time(df = tmp,
                          var1 = 'Time',
                          var2 = 'asynch',
                          var3 = 'Duration',
                          ylabel = 'Asynchrony (ms)')
fig9

Synchrony across Several Performances (corpus)

corpus <- onsetsync::CSS_IEMP                              # This is a small corpus
D <- sync_sample_paired(corpus,'Tres','Bass',N=0,beat='SD')
D <- D$asynch
D$asynch_abs <- abs(D$asynch)*1000                         # Convert to ms
fig7 <- plot_by_dataset(D,'asynch_abs','name', box = TRUE)
print(fig7)

Periodicity

Periodicity of Guitar Onsets

extract <- dplyr::filter(CSS_Song2, Guitar >= 60 & Guitar < 65)
fig10 <- gaussify_onsets(extract$Guitar, wlen = 0.2, plot = TRUE)
print(fig10$fig)

Estimate Periodicity

Four methods to estimate periodicity (autocorrelation, FFT, smoothed FFT, or difference)

per <- periodicity(extract,instr='Guitar',freq_range=c(0.1,0.6))
print(per$Figure)

Summarise Periodicity

Convenience functions to extract peak of periodicity curve.

psumm <- summarise_periodicity(per$Curve)
print(paste('Period is:', round(psumm$Per,3), 'seconds'))

Or convert to BPM

print(paste('In BPM:',period_to_BPM(psumm$Per*2))) # knowing these are sub-beats

Estimate Periodicity

Use autocorrelation this time...

per2 <- periodicity(extract,instr='Guitar',method='acf',freq_range=c(0.1,0.6))
print(per2$Figure)

Other Sources of Onset Data {.smaller}

Real performances

• CompMusic datasets at UPF: Hindustani (North India), Carnatic (South India), Turkish-makam (Turkey), Arab-Andalusian (Maghreb), and Beijing Opera (China)
• Tap & Fiddle: Scandinavian Fiddle Tunes with Accompanying Foot-Tapping

• The MAESTRO (MIDI and Audio Edited for Synchronous Tracks and Organisation)

  • 200 hours of real recordings with MIDI and audio (Piano competition)

• University of Rochester Multi-Modal Musical Performance

• Carnatic Music Rhythm Dataset

Credits

::: callout-note Thanks for Adrian Poole and Simone Tarsitani for preparing IEMP Cuban data included here. :::

References

Clayton, M., K. Jakubowski, T. Eerola, P. Keller, A. Camurri, G. Volpe, and P. Alborno. (2020). Interpersonal Entrainment in Music Performance: Theory, Method and Model. Music Perception 38 (2), 136–94.

Clayton, M., S. Tarsitani, R. Jankowsky, L. Jure, L. Leante, R. Polak, A. Poole, et al. (2021). The Interpersonal Entrainment in Music Performance Data Collection. Empirical Musicology Review 16 (1), 65–84. http://dx.doi.org/10.18061/emr.v16i1.7555

Rasch, Rudolf A. (1979). Synchronization in Performed Ensemble Music. Acta Acustica United with Acustica 43 (2), 121–31.

Repp, Bruno H, and Yi-Huang Su. (2013). Sensorimotor Synchronization: A Review of Recent Research (2006–2012). Psychonomic Bulletin & Review 20 (3), 403–52.

Wilkinson, Mark D, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, et al. (2016). The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific Data 3 (1), 1–9.

Wing, Alan M, Satoshi Endo, Adrian Bradbury, & Dirk Vorberg. (2014). Optimal Feedback Correction in String Quartet Synchronization. Journal of The Royal Society Interface 11 (93), 20131125.

tuomaseerola/onsetsynch documentation built on Jan. 17, 2024, 10:58 a.m.