[PYTHON] [Voice analysis] Find Cross Similarity with Librosa

What is Cross Similarity?

A pattern extraction method specialized for partial matching, which is a derivative of ** Dynamic Time Stretching Method (DTW) **. DTW is a metric that creates a distance matrix between time series data, finds one minimum path and uses it as the distance, but Cross Similarity finds multiple paths for partial matching. An easy-to-understand explanation about DTW is here

For Cross Similarity, see the paper below. Discovery of Cross Similarity in Data Science(Toyoda et al. 2010)

Why Cross Similarity

When I wanted to segment only the song from the long-term data of the bird's bark, the waveform of the bark has the characteristic of repeating the typology, so I thought that it would be suitable for this method, so I experimented.

environment

python 3.7.4 librosa 0.7.2 matplotlib 3.1.1

Let's Librosa! We used the simplest example from the Librosa docs. I'm studying Cross Similarity variables, parameters, algorithms, etc ...

Calculation of Cross Similarity

import librosa
import librosa.feature
import librosa.segment
import librosa.display

hop_length = 1024
y_ref, sr = librosa.load("path to sound file")
y_comp, sr = librosa.load("path to sound file")
chroma_ref = librosa.feature.chroma_cqt(y=y_ref, 
  sr=sr,hop_length=hop_length)
chroma_comp = librosa.feature.chroma_cqt(y=y_comp, 
  sr=sr, hop_length=hop_length)

x_ref = librosa.feature.stack_memory(
  chroma_ref, n_steps=10, delay=3)
x_comp = librosa.feature.stack_memory(
  chroma_comp, n_steps=10, delay=3)
xsim = librosa.segment.cross_similarity(x_comp, x_ref)

plot

import matplotlib.pyplot as plt
fig, ax = plt.subplots()

librosa.display.specshow(xsim, x_axis='s', y_axis='time', hop_length=hop_length, ax=ax)
plt.show()

I think the image will look like a document. that's all. Thank you for reading.