Verification 5) Check how many clusters are valid

• 4 clusters to match the 4 characters registered as kanji.
• Then, looking at the results of 4 clusters, I tried to separate them into 8 clusters.
• As expected, ** how many clusters is appropriate? ** **
• Here, as a method of selecting the number of clusters, I would like to visualize and examine the cluster status using the following three methods.

1. Elbow Chart
2. Silhouette Chart
3. dendrogram

Elbow Chart

• The elbow chart is a chart with ** data variation in each cluster ** on the vertical axis and ** number of clusters ** on the horizontal axis.
• Increasing the number of clusters can reduce the variability, but too many clusters is also a problem.
• Therefore, ** the number of clusters ** is reasonable, and ** the number of clusters that can reduce data variability ** is considered in this figure.
• Yellowbrick will be used for drawing.

from yellowbrick.cluster import KElbowVisualizer
vis = KElbowVisualizer(
    KMeans(random_state=42),
    k=(1,34) #Number of clusters (range on the horizontal axis))
)
vis.fit(embeddings)
vis.show()

• The feeling you see is
• Up to 5 clusters, ** data variability (average) decreased **, but thereafter it became flat.
• Therefore, ** classified into 5 clusters ** seems to be good = the representative kanji is ** 5 types ** seems to be good.
• But let's take a look at the enlarged version (enlarged by 4-18)
• There seems to be an inflection point at 5, but it has become flat from ** around 10 **.
• In other words, it seems that there is no mistake in ** classifying into 8 clusters and deciding 8 representative kanji **.

from yellowbrick.cluster import KElbowVisualizer
vis = KElbowVisualizer(
    KMeans(random_state=42),
    k=(4,19) #Number of clusters (range on the horizontal axis))
)
vis.fit(embeddings)
vis.show()

Silhouette Chart

• Silhouette Chart is a chart that expresses the following for each cluster.
• Vertical axis (thickness of graph): Number of samples in the cluster
• Horizontal axis (graph length): Silhouette coefficient of the cluster
• Dashed line: Average silhouette coefficient
• From the perspective, the point is to find the number of Glasta that satisfies the following.
• Same number of samples for all clusters = same thickness
• Silhouette coefficient is close to average for all Glasta = Length is close to broken line
• We will also use Yellowbrick for drawing.

from yellowbrick.cluster import silhouette_visualizer
fig = plt.figure(figsize=(15,25))
#Draw together from 4 to 9 clusters
for i in range(4,10):
    ax = fig.add_subplot(4,2,i-1)
    silhouette_visualizer(KMeans(i),embeddings)

• As you can see, the pattern on the upper right (** clusters 5 **) is nice.

dendrogram

• It is a graph that expresses ** closeness ** between clusters like a tournament table.
• Since it is a diagram that can be used in hierarchical clustering, Scipy's hierarchical clustering is used instead of KMeans.
• The view is as follows.
• Clusters with leaves as data and branches of the same color with the same range
• Height is the distance between clusters

from scipy.cluster.hierarchy import linkage, dendrogram
Z = linkage(
    y = embeddings,
    method = 'weighted',
    metric = "euclidean",
)

R = dendrogram(
    Z=Z,
    color_threshold=1.2, #Adjust the number of clusters with this threshold
    show_contracted=False,
)

• ** It would be nice if the number of branches of each color is well-balanced and the heights are the same. After all, is the number of clusters about 5?

Number of clusters	Dendrogram	comment
４		RedJust a little expensive
５		The height is uniform purpleI'm worried about a few It feels pretty good
８		The height and number are the same, Is it divided too finely?

Verification 6) Try to make 5 clusters

• Since I examined the number of clusters, I would like to plot again what it will look like with 5 clusters.
• Sounds pretty good. After all, is it 5 clusters?

from sklearn.cluster import KMeans
# clustering(Number of clusters: 5)
clustering = KMeans(n_clusters=5, random_state=42)
# fit_predict cluster
cl_y = clustering.fit_predict(embeddings)
# visualize
showScatter(embeddings, cl_y, clustering.cluster_centers_)

No	cluster	Center of gravity	Other characters included
1	Blue
2	purple
3	Green
4	Red
5	orange

Summary

Impressions

• As a flow,
• Start by selecting a representative of the 4 characters registered as kanji
• For all 33 characters that are not registered as kanji, select 1, 4 or 8 characters.
• Considering the appropriate number of clusters, I chose 5 characters at the end because 5 clusters seemed to be good.
• The representative kanji are as follows, but more than deciding the representative
• It is also interesting that on the XY plane where 3000 dimensions are compressed, ** kanji with similar shapes are placed nearby **.
• It was interesting to be able to create ** groups by radical ** with distance-based clustering.
• The number of clusters was also judged to be 5 clusters based on the results of the elbow method, silhouette method, and dendogram.
• It was also interesting that the ** results of the clustering visualization of 5 clusters were reasonably good **.

Verification list

No	How to choose	Representative Saito
1	From the 4 recognized kanji 1 characterIf you choose, the representative is
2	From all 33 kanji 1 characterIf you choose
3	From all 33 kanji 4 charactersIf you choose
4	From all 33 kanji 8 charactersIf you choose
5	All 33 kanji How many clustersShould be divided into	About 5 clustersLooks good
6	From all 33 kanji 5 charactersIf you choose

Finally

• Thank you for dealing with such a silly story.
• If you like, I'd appreciate it if you could share it.

Reference information

• About UMAP
  • https://umap-learn.readthedocs.io/en/latest/index.html
• Discussion about Clustering in UMAP (seems to be)
  • https://umap-learn.readthedocs.io/en/latest/index.html
• About KMeans
  • https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
• Examining the number of clusters (Yellow Brick)
  • https://www.scikit-yb.org/en/latest/api/cluster/elbow.html
  • https://www.scikit-yb.org/en/latest/api/cluster/silhouette.html
• Drawing with dendrogram
  • https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.cluster.hierarchy.dendrogram.html

Visualization function

• I referred to this article. Thank you very much. I will link.

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
from matplotlib import offsetbox
from sklearn.preprocessing import MinMaxScaler
from PIL import Image
import matplotlib.patches as patches

rc = {
  'font.family': ['sans-serif'],
  'font.sans-serif': ['Open Sans', 'Arial Unicode MS'],
  'font.size': 12,
  'figure.figsize': (8, 6),
  'grid.linewidth': 0.5,
  'legend.fontsize': 10,
  'legend.frameon': True,
  'legend.framealpha': 0.6,
  'legend.handletextpad': 0.2,
  'lines.linewidth': 1,
  'axes.facecolor': '#fafafa',
  'axes.labelsize': 10,
  'axes.titlesize': 14,
  'axes.linewidth': 0.5,
  'xtick.labelsize': 10,
  'xtick.minor.visible': True,
  'ytick.labelsize': 10,
  'figure.titlesize': 14
}
sns.set('notebook', 'whitegrid', rc=rc)

def colorize(d, color, alpha=1.0):
  rgb = np.dstack((d,d,d)) * color
  return np.dstack((rgb, d * alpha)).astype(np.uint8)

colors = sns.color_palette('tab10')

def showScatter(
    embeddings,
    clusterlabels,
    centers = [],
    imgs = all.T.reshape(-1,h,w),
):
    fig, ax = plt.subplots(figsize=(15,15))
    
    #Scaling before drawing scatter plot
    scaler = MinMaxScaler()
    embeddings = scaler.fit_transform(embeddings)
    
    source = zip(embeddings, imgs ,clusterlabels)
    
    #Draw kanji on a scatter plot
    cnt = 0
    for pos, d , i in source:
        cnt = cnt + 1
        img = colorize(d, colors[i], 0.5)
        ab = offsetbox.AnnotationBbox(offsetbox.OffsetImage(img),0.03 + pos * 0.94,frameon=False)
        ax.add_artist(ab)
          
    #Draw concentric circles from the center of gravity
    if len(centers) != 0:
        for c in scaler.transform(centers):
            for r in np.arange(3,0,-1)*0.05:
                circle = patches.Circle(
                    xy=(c[0], c[1]),
                    radius=r,
                    fc='#FFFFFF', 
                    ec='black'
                )
                circle.set_alpha(0.3)
                ax.add_patch(circle)

            ax.scatter(c[0],c[1],s=300,marker="X")
  

    #Axis drawing range
    limit = [-0.1,1.1]
    plt.xlim(limit)
    plt.ylim(limit)
    plt.show()