[PYTHON] I vectorized the chord of the song with word2vec and visualized it with t-SNE

So, it's a complicated process, but let's take a look at the code.

scraping.py

import csv
import glob
import collections
import re
import os

artist = "aiko"
#Data directory
fdir = "chord_data/" + artist + "/"
os.makedirs(fdir,exist_ok=True)
#Key type
key_list = ["C","C#","D","D#","E","F","F#","G","G#","A","A#","B"]
#Roman numerals
rome_list = ["Ⅰ","#I","Ⅱ","#Ⅱ","Ⅲ","Ⅳ","#Ⅳ","Ⅴ","#Ⅴ","Ⅵ","#Ⅵ","Ⅶ"]
#Diatonic code
dtnc_chord_list_num = ["Ⅰ","Ⅱm","Ⅲm","Ⅳ","Ⅴ","Ⅵm","Ⅶm7-5"]
#All all half all all half
dtnc_step = [2,2,1,2,2,2,1]
#♭#Convert to
r_dict = {'D♭':'C#', 'E♭':'D#', 'G♭':'F#','A♭':'G#','B♭':'A#'}
#Chord progression loading
flist = glob.glob(fdir+"*")

dtnc_chord_list_arr = []

for idx,key in enumerate(key_list):
    pos = idx
    dtnc_chord_list = []
    for num,step in enumerate(dtnc_step):
        #root sound + major or minor
        dtnc_chord_list.append(key_list[pos]+dtnc_chord_list_num[num][1:])
        pos += step
        #Express pos as a number of 12 or less
        if pos >= len(key_list):
            pos -= len(key_list)
    #Stores a list of diatonic codes for 12 different keys
    dtnc_chord_list_arr.append(dtnc_chord_list)


for fname in flist:
    with open(fname,encoding='cp932',mode='r+') as f:
        f.readline().rstrip('\n')
        chord_list = f.readline().rstrip('\n')
        f.readline().rstrip('\n')
        lyric_list = f.readline().rstrip('\n')
        #♭#Convert to
        chord_list = re.sub('({})'.format('|'.join(map(re.escape, r_dict.keys()))), lambda m: r_dict[m.group()], chord_list)
        chord_list = chord_list.split(',')
        chord_list_origin = chord_list.copy()
        # N.C.Get rid of
        chord_list = [chord for chord in chord_list if "N" not in chord]
        #Triad only
        def get_triad(chord):
            split_chord = list(chord)
            triad = split_chord[0]
            if len(split_chord )>=2 and split_chord[1]=='#':
                triad += split_chord[1]
                if len(split_chord )>=3 and split_chord[2]=='m':
                    triad += split_chord[2]
                    if len(split_chord )>=5 and split_chord[4]=='-':
                        triad += split_chord[3]
                        triad += split_chord[4]
                        triad += split_chord[5]

            elif len(split_chord )>=2 and split_chord[1]=='m':
                triad += split_chord[1]
                if len(split_chord )>=4 and split_chord[3]=='-':
                        triad += split_chord[2]
                        triad += split_chord[3]
                        triad += split_chord[4]
            else:
                pass

            return triad
        #Change to triad
        chord_list_triad = [get_triad(chord) for chord in chord_list]
        length = len(chord_list)
        #Unique count of code
        chord_unique = collections.Counter(chord_list_triad)
        #print(chord_unique)

        #################
        ###Determine the key###
        #################

        match_cnt_arr = []
        #Calculate the number of matches with 12 types of keys
        for dtnc_chord_list in dtnc_chord_list_arr:
            match_cnt = 0
            #Chord each of the 7 diatonic codes_Compare with the unique key value,
            #If there is a match, match_Add to cnt
            for dtnc_chord in dtnc_chord_list:
                if dtnc_chord in chord_unique.keys():
                    match_cnt += chord_unique[dtnc_chord]
            match_cnt_arr.append(match_cnt)
        #Maximum number of matches among 12 types
        max_cnt = max(match_cnt_arr)
        #Number of matched codes/Total number of codes(%)
        match_prb = int((max_cnt/length)*100)

        #Key determination
        key_pos = match_cnt_arr.index(max_cnt)
        key = key_list[key_pos]
        dtnc_chord_list = dtnc_chord_list_arr[key_pos]
        file_name = os.path.basename(fname).replace('.csv','')
        print('Song title:{0} , key:{1} , prob:{2}'.format(file_name,key,match_prb))
        print(dtnc_chord_list)
        key_list_chromatic = key_list[key_pos:] + key_list[:key_pos]
        # key_list_chromatic.extend(key_list[key_pos:])
        # key_list_chromatic.extend(key_list[:key_pos])
        print(key_list_chromatic)

        #Convert code to Roman numerals and write to file
        #Function to convert
        def convert_num_chord(chord_list):
            s_list = []
            n_list = []
            for idx, root in enumerate(key_list_chromatic):
                #Since the one with a sharp is replaced first, the root is divided according to the presence or absence of a sharp.
                if '#' in root:
                    s_list.append([idx,root])
                else:
                    n_list.append([idx,root])
            chord_list = ['*' if "N" in chord else chord for chord in chord_list]
            for idx,root in s_list:
                chord_list = [chord.replace(root,rome_list[idx]) if root in chord else chord for chord in chord_list]
            for idx,root in n_list:
                chord_list = [chord.replace(root,rome_list[idx]) if root in chord else chord for chord in chord_list]
            chord_list = ['N.C.' if "*" in chord else chord for chord in chord_list]
            
            return chord_list

        chord_list_converted = convert_num_chord(chord_list_origin)
        print(chord_list_origin)
        print(chord_list_converted)
    with open(fname, "w", encoding="cp932") as f:
        writer = csv.writer(f)
        writer.writerow('key:{0},prob:{1}'.format(key,match_prb).split(','))
        writer.writerow(chord_list_origin)
        writer.writerow(chord_list_converted)
        writer.writerow(lyric_list.split(','))

How was that. It's long and hard to understand what you're doing, but you can recognize that you've standardized it. It's not a big deal, so it's done in an instant. When I run this code, I get the following results:

You can see that key = D # on the first line, the probability of belonging is 67%, and the chord progression of Roman numerals after standardization is added on the third line. (Since the code of aiko is quite complicated and many non-diatonic codes appear, the probability of belonging to diatonic is 67%. Even with this, the key can be estimated correctly, so the correctness of this process is correct. I think you can prove it.)

Now, in the last chapter, let's vectorize and draw this standardized code.

Chapter 3: Vectorize the code with word2vec and show it with t-SNE

Now, let's put the data of about 200 songs of aiko into word2vec and vectorize it. The number of dimensions is set to 10 because there are at most 100 types of output code. Also, for window, the role of the code can be judged sufficiently with 4 codes before and after, so I specified it as 4. In addition, sg = 0 was set in order to adopt CBOW (Continuous Bag-of-Words) that estimates the central word from the periphery. This transforms each code into a 10-dimensional vector. In order to see this result as a figure, it was compressed in two dimensions using t-SNE and shown in the figure. The perplexity setting here is a difficult place, but I set it to 3 because the smaller one is easier to separate for each cluster.

If it's about compressing the dimensions, why not make the representation dimension of the code two-dimensional? I'm sure there are people who think that, but then there is almost no difference in the code, so I made it 10 dimensions to put it on the expressiveness.

Now let's look at the code.

analyze.py

from gensim.models import Word2Vec
import glob
import itertools
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE

artist = "aiko"
#Data directory
fdir = "chord_data/" + artist + "/"
#Chord progression loading
flist = glob.glob(fdir+"*")
music_list = []
for fname in flist:
    with open(fname,encoding='cp932',mode='r+') as f:
        f.readline().rstrip('\n')
        f.readline().rstrip('\n')
        chord_list = f.readline().rstrip('\n').split(',')
        f.readline().rstrip('\n')
        lyric_list = f.readline()
        chord_list = [chord for chord in chord_list if "N" not in chord]
        music_list.append(chord_list)

#Vectorize code
model = Word2Vec(music_list,sg=0,window=4,min_count=0,iter=100,size=10)
chord_unique = list(set(itertools.chain.from_iterable(music_list)))
data = [model[chord] for chord in chord_unique]
print(model.most_similar('Ⅱ'))


#Compress to 2D and draw
tsne = TSNE(n_components=2, random_state = 0, perplexity = 3, n_iter = 1000)
data_tsne = tsne.fit_transform(data)

fig=plt.figure(figsize=(50,25),facecolor='w')

plt.rcParams["font.size"] = 10

for i,chord in enumerate(data_tsne):
    #Point plot
    plt.plot(data_tsne[i][0], data_tsne[i][1], ms=5.0, zorder=2, marker="x",color="red")
    plt.annotate(chord_unique[i],(data_tsne[i][0], data_tsne[i][1]), size=10)
    i += 1

#plt.show()
plt.savefig("chord_data/" + artist + ".jpg ")
 

The result obtained with this code is as follows.

Aiko has many types of code and it is difficult to find a cluster, For example, in the lower left circle from the center, the triads I, IV, and V are lined up nearby. This is also correct in music theory, and this list of chord combinations is the most frequently occurring chord, and because of its relevance, the vectors are probably closer. In the same circle, there is a code called II, but although this is a non-diatonic code, I think that it was likely that it existed nearby as a doppel dominant in front of the code V. There are also # V and # VI, which are often used in the famous progression of # V → # VI → I, and it is thought that these two codes are close to each other. Besides, there is a set of codes with 7th in the upper circle, so I think that it was an interesting result to see.

Since it's a big deal, I will post the results of other artists as well. First, the following is Official Hige Dandism.

And the other is an artist called andymori. This may be easier to read because it uses less code.

Conclusion and future prospects

In conclusion, I found that putting the code in word2vec leads to some meaningful data.

As a future prospect, let's remember the famous chord progression in advance, and divide the song by about 4 chord progressions for that. It may be interesting to create a sparse matrix as a chord progression, create a topic in LDA, and determine the similarity of songs within an artist. It would be even better if we could even make recommendations between artists. Also, for those who are only interested in music, I would like to have a tool that makes it easy to see where the modulation is due to the work in Chapter 2.

If you are writing an article that combines music and programming, please make friends. ..

Thank you for reading.