Introduction

** * This article is intended for those who have programming experience in Python. ** **

In addition, the following knowledge is assumed. ・ Basic grammar of Python ・ Data visualization with matplotlib

Purpose

Obtains a card list from the JCG site and automatically aggregates and graphs class distributions and archetype distributions.

approach

On the JCG site, registrant information such as tournament winning information, check-in status, registration deck, etc. are summarized in a text file in JSON format. Get the card list from this JSON file and sort the deck types from the number composition.

What is JSON

JSON (JavaScript Object Notation) is a notation for expressing structured data. Imagine that it represents a hierarchical structure between texts, like a hierarchical structure of directories. For example, in the JCG JSON file, the registrant data is described in the following format.

`example.json`


{
	"result": "success",
	"participants": [
		{
			"id": "42155", 
			"chk": 1, #Check-in status
			"nm":  "somey", #Registered name
			"dk": [ #Deck information
				{
					"cl": 3, #Class 1
					"hs": "3.3.6so2A.6so2A.5-gkQ.5-gkQ.5-gkQ.6_X7Q.6_X7Q.6_X7Q.6_djc.6_djc.6_djc.6turQ.6turQ.6turQ.6xnsw.6xnsw.6xnsw.6_ZZg.6_ZZg.6_ZZg.gHqNk.gHqNk.gHqNk.6ty_2.6ty_2.6ty_2.6q8sC.6q8sC.6q8sC.6twYy.6twYy.6twYy.6xpaI.6xpaI.5-glM.5-glM.5-glM.6t_RI.6t_RI.6t_RI"
				}, #Deck 1 card list (to be exact, part of the portal URL)
				{
					"cl": 8, #Class 2
					"hs": "3.8.71TeA.71TeA.71TeA.6zemS.6zemS.6zemS.71QTC.71QTC.71QTC.6zd2w.6zd2w.6zd2w.6s2wi.6zcK2.6zcK2.6zcK2.6s65g.6zcKC.6zcKC.6zcKC.6s5My.6s5My.6s5My.6zhCY.6zhxQ.6zhxQ.6zhxQ.6-UTo.6-UTo.71Xo6.71Xo6.71Xo6.6oEnY.6oEnY.6oEnY.6oHDo.6oHDo.6vvW6.6vvW6.6vvW6"
				} #Deck 2 card list (to be exact, part of the portal URL)
			],
			"en": 1527921725,
			"te": 1, #Winning information
			"pr": 1,
			"cu": 0
		},

Acquisition of deck information

Information about the deck is stored in the "hs" key part of the JSON file.

For example, in the JSON file above

`6so2A.6so2A.5-gkQ.5-gkQ.5-gkQ.6_X7Q.6_X7Q.6_X7Q.6_djc.6_djc.6_djc.6turQ.6turQ.6turQ.6xnsw.6xnsw.6xnsw.6_ZZg.6_ZZg.6_ZZg.gHqNk.gHqNk.gHqNk.6ty_2.6ty_2.6ty_2.6q8sC.6q8sC.6q8sC.6twYy.6twYy.6twYy.6xpaI.6xpaI.5-glM.5-glM.5-glM.6t_RI.6t_RI.6t_RI`


 Is the deck information.

 Five-digit alphanumeric characters (such as `` `6so2A```) separated by periods represent one card, and 40 of them together represent one deck.
 Check the Shadowverse portal for the correspondence between this alphanumeric character and the card name.
 If you take a closer look at the URL of the portal, you'll see that it's made up of 40 alphanumeric characters, similar to a JSON file.

 (Example: spell witch)
https://shadowverse-portal.com/deckbuilder/create/3?hash=3.3.6so2A.6so2A.5-gkQ.5-gkQ.5-gkQ.6_X7Q.6_X7Q.6_X7Q.6_djc.6_djc.6_djc.6turQ.6turQ.6turQ.6xnsw.6xnsw.6xnsw.6_ZZg.6_ZZg.6_ZZg.gHqNk.gHqNk.gHqNk.6ty_2.6ty_2.6ty_2.6q8sC.6q8sC.6q8sC.6twYy.6twYy.6twYy.6xpaI.6xpaI.5-glM.5-glM.5-glM.6t_RI.6t_RI.6t_RI&lang=ja

 For example, three alphanumeric characters ``` 6t_RI``` appear in succession at the end, which means that three "Yin and Yang founders, Quon" are included.

### Archetype classification

 Create a judgment formula to classify archetypes using information on the type and number of cards included in the deck.
 For the judgment formula, use a card that can uniquely identify the archetype as much as possible.

 Example: `` `6q95g``` (magic tool specialty store) is included 3 times → Specialty store witch

### Analysis procedure
 1. Request a json file
 (Hereafter analysis of json file)
 2. Confirmation of winning information (Go to 3 if a winning participant is found)
 3. Get the card list
 4. Archetype classification (If ✕✕ is ○ or more, □□ elf, etc.)
 5. Repeat steps 2-4
 6. After the archetype classification of all participants is completed, aggregate and graph

### An example of aggregation results

 ** JCG Shadowverse Open 14th Season Vol.29 August 22 Rotation Tournament Group Qualifying **

 ** Class distribution **
 <img width="480" alt="class_pie_2328.png " src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/693609/00e0c41e-4f70-ed10-627e-105cd08fa0f1.png ">
 ** Archetype distribution **
 <img width="480" alt="class_bar_2328.png " src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/693609/14f786cb-6953-989a-89cf-5bbaa30cc21d.png ">

# coding
 * Because you are self-taught about programming, you may not follow redundant code or customs.
 Please keep the coding examples below for reference only.
### Library import
 First import the required libraries.


#### **`import.py`**
```python

import requests
import json

Counter definition

Defines the variables used to aggregate the number of classes and archetypes. It also defines a two-dimensional dictionary variable arche_dict to associate the number of classes, the number of archetypes, and the archetype name.

The archetype variables and archetype names in the code below are examples at the time of writing the article (August 22, 2020). If the environment changes significantly due to the timing of new bullets or additional card mounting, please rewrite as appropriate according to the new environment.

`counter.py`


#Definition of class counter
E = R = W = D = Nc = V = B = Nm = 0
#Archetype counter definition
E1 = E2 = R1 = R2 = W1 = W2 = W3 = D1 = D2 = Nc1 = Nc2 = V1 = V2 = B1 = B2 = Nm1 = 0
#Other archetype counters
OE = OR = OW = OD = ONc = OV = OB = ONm = 0

#Dictionary of class counters, archetype names, archetype counters
arche_dict = {"E":{"Reno Seus E":E1, "Amatsu E":E2, "Other E":OE},"R": {"Evolution R":R1, "Cooperation R":R2, "Other R":OR},"W": {"Spell W":W1, "Specialty store W":W2, "Arcane W":W3, "Other W":OW},"D": {"Discard D":D1, "Whale D":D2, "Other D":OD},"Nc": {"Netherworld Nc":Nc1, "Funeral Nc":Nc2,  "Other Nc":ONc},"V": {"Control V":V1, "Frenzy V":V2, "Other V":OV},"B": {"Eira B":B1, "Control B":B2, "Other B":OB},"Nm": {"AFNm":Nm1, "Other Nm":ONm}}

Archetype judgment formula

Defines a function that determines the archetype. It receives class information and a list of 40 cards as arguments, and sorts them in the order of classification → archetype classification.

An example of the judgment formula is shown below. In order to improve the accuracy of classification, let's set an appropriate judgment formula through trial and error.

`json_request.py`


#Class, archetype analysis
def deck_arche_analysis(sv_deck, sv_class):
    global E,R,W,D,Nc,V,B,Nm
    global E1,E2,R1,R2,W1,W2,W3,D1,D2,Nc1,Nc2,V1,V2,B1,B2,Nm1
    global OE,OR,OW,OD,ONc,OV,OB,ONm
    if sv_class == 1: #Elf
        E += 1
        if sv_deck.count("6lZu2") == 3:
            E1 += 1
        elif sv_deck.count("6pQTI") == 3:
            E2 += 1
        else:
            OE += 1
    elif sv_class == 2: #Royal
        R += 1
        if sv_deck.count("6td16") > 1:
            R1 += 1
        elif sv_deck.count("6_B9A") == 3:
            R2 += 1
        else:
            OR += 1
    elif sv_class == 3: #Witch
        W += 1
        if sv_deck.count("6_djc") == 3:
            W1 += 1
        elif sv_deck.count("6q95g") == 3:
            W2 += 1
        elif sv_deck.count("6t_Rc") == 3:
            W3 += 1
        else:
            OW += 1
    elif sv_class == 4: #Dragon
        D += 1
        if sv_deck.count("6yB-y") == 3:
            D1 += 1
        elif sv_deck.count("6_zhY") == 3:
            D2 += 1
        else:
            OD += 1
    elif sv_class == 5: #Necromancer
        Nc += 1
        if sv_deck.count("6n7-I") > 1:
            Nc1 += 1
        elif sv_deck.count("70OYI") == 3:
            Nc2 += 1
        else:
            ONc += 1
    elif sv_class == 6: #Vampire
        V += 1
        if sv_deck.count("6rGOA") == 3:
            V1 += 1
        elif sv_deck.count("6v1MC") ==3:
            V2 += 1
        else:
            OV += 1
    elif sv_class == 7: #Bishop
        B += 1
        if sv_deck.count("6nupS") == 3:
            B1 += 1
        elif sv_deck.count("6nsN2") == 3:
            B2 += 1
        else:
            OB += 1
    elif sv_class == 8: #Nemesis
        Nm += 1
        if sv_deck.count("6zcK2") == 3:
            Nm1 += 1
        else:
            ONm += 1

Get JSON file

`json_request.py`


#Enter the JCG tournament number
compe_num = input("Please enter the JCG tournament number you want to look up")
#URL of the tournament json file
jcg_url = "https://sv.j-cg.com/compe/view/entrylist/" +  str(compe_num) + "/json"
#Request json file
res_jcg = requests.get(jcg_url)
#Save as json format data
j_txt = json.loads(res_jcg.text)

Parse JSON file and aggregate classes and archetypes

`data-processing.py`


for i in range(len(j_txt["participants"])):
    #Confirmation of winning information
    if j_txt["participants"][i]["te"] == 0: #Lost
        continue
    elif j_txt["participants"][i]["te"] == 1: #Winning
        for j in range(2):
            #Get class information
            class_ij = j_txt["participants"][i]["dk"][j]["cl"]
            #Acquisition of card information
            deck_ij = j_txt["participants"][i]["dk"][j]["hs"]
            #Determine archetype from class card information
            deck_arche_analysis(deck_ij, class_ij)
    else:
        continue

Format data for graphs

`data.py`


#Reflection of aggregation results
arche_dict = {"E":{"Reno Seus E":E1, "Amatsu E":E2, "Other E":OE},"R": {"Evolution R":R1, "Cooperation R":R2, "Other R":OR},"W": {"Spell W":W1, "Specialty store W":W2, "Arcane W":W3, "Other W":OW},"D": {"Discard D":D1, "Whale D":D2, "Other D":OD},"Nc": {"Netherworld Nc":Nc1, "Funeral Nc":Nc2,  "Other Nc":ONc},"V": {"Control V":V1, "Frenzy V":V2, "Other V":OV},"B": {"Eira B":B1, "Control B":B2, "Other B":OB},"Nm": {"AFNm":Nm1, "Other Nm":ONm}}

#Class count, label array
class_count = [E, R, W, D, Nc, V, B, Nm]
class_name = ["E", "R", "W", "D", "Nc", "V", "B", "Nm"]

#Archetype count, label array
count = [list(arche_dict[key].values()) for key in arche_dict]
arche_count = sum(count,[])
label = [list(arche_dict[key].keys()) for key in arche_dict]
arche_name = sum(label,[])

Graphing

Finally, graph the analysis results. An example of the plot is shown below, but please try to graph it with your favorite layout.

** * If you want to execute the following code as it is, you need to set the Japanese font in matplotlib. ** ** If you want to execute it as it is, download the font referring to the following article and place it in the same directory as this python file.

https://tech-k-labs.xyz/post/others/matplotlib_with_heroku/ (Graph drawing of matplotlib on Heroku)

`graphs.py`


#Import matplotlib
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
#The following two lines are for Japanese setting
from matplotlib.font_manager import FontProperties
fontprop = FontProperties(fname="ipaexg.ttf")

#Color settings
class_colors = ["palegreen", "peachpuff", "mediumslateblue", "sienna","darkmagenta", "crimson", "wheat", "lightsteelblue"]
arche_colors = ["palegreen"]*len(arche_dict["E"]) +["peachpuff"]*len(arche_dict["R"]) +  ["mediumslateblue"] * len(arche_dict["W"]) + ["sienna"] * len(arche_dict["D"]) + ["darkmagenta"] * len(arche_dict["Nc"]) + ["crimson"] * len(arche_dict["V"]) + ["wheat"] * len(arche_dict["B"]) + ["lightsteelblue"] * len(arche_dict["Nm"])

#Pie chart of class distribution
fig1 = plt.figure()
plt.pie(class_count, labels=class_name, colors=class_colors, autopct="%.1f%%",pctdistance=1.35,wedgeprops={'linewidth': 2, 'edgecolor':"white"})
fig1.savefig("class_pie_"+compe_num+".png ")

#Bar chart of archetype distribution
fig2 = plt.figure()
#x = np.array(list(range(len(arche_name))))
x = list(range(len(arche_name)))
plt.bar(x, arche_count, color=arche_colors)
plt.ylabel("Number of use",font_properties=fontprop)
plt.xticks(x,arche_name,rotation=90,font_properties=fontprop)
plt.subplots_adjust(left=0.1, right=0.95, bottom=0.25, top=0.95)
for x, y in zip(x, arche_count):
    plt.text(x, y, y, ha='center', va='bottom')

fig2.savefig("class_bar_"+compe_num+".png ")

Execution result

Enter the 4-digit tournament number at the end of the JCG URL and execute. (The URL of the tournament examined this time: https://sv.j-cg.com/compe/2328)

** Command line ** ** Class distribution ** ** Archetype distribution **

in conclusion

This time, I explained the archetype analysis of the rotation environment as an example. Unlimited environment can be analyzed by changing the judgment logic, so if you are interested, please try it.

Reference 1: Comparison of the number of (development) cards adopted

You can also automatically create a comparison table of the number of hires by acquiring the card name and the number of cards from the shadowverse portal by web scraping.

** Example: JCG Shadowverse Open 14th Season Vol.29 August 22nd Rotation Tournament Final Tournament ** ** Spell Witch **

More on this in another article if I have the opportunity.

Reference 2: (Development) Execute from Discord

It is also possible to execute programs from Discord by using Discord bot.

Automatically aggregate JCG deck distribution with Python

Introduction

Purpose

approach

What is JSON

example.json

Acquisition of deck information

6so2A.6so2A.5-gkQ.5-gkQ.5-gkQ.6_X7Q.6_X7Q.6_X7Q.6_djc.6_djc.6_djc.6turQ.6turQ.6turQ.6xnsw.6xnsw.6xnsw.6_ZZg.6_ZZg.6_ZZg.gHqNk.gHqNk.gHqNk.6ty_2.6ty_2.6ty_2.6q8sC.6q8sC.6q8sC.6twYy.6twYy.6twYy.6xpaI.6xpaI.5-glM.5-glM.5-glM.6t_RI.6t_RI.6t_RI

Counter definition

counter.py

Archetype judgment formula

json_request.py

Get JSON file

json_request.py

Parse JSON file and aggregate classes and archetypes

data-processing.py

Format data for graphs

data.py

Graphing

graphs.py

Execution result

in conclusion

Reference 1: Comparison of the number of (development) cards adopted

Reference 2: (Development) Execute from Discord

`example.json`

`6so2A.6so2A.5-gkQ.5-gkQ.5-gkQ.6_X7Q.6_X7Q.6_X7Q.6_djc.6_djc.6_djc.6turQ.6turQ.6turQ.6xnsw.6xnsw.6xnsw.6_ZZg.6_ZZg.6_ZZg.gHqNk.gHqNk.gHqNk.6ty_2.6ty_2.6ty_2.6q8sC.6q8sC.6q8sC.6twYy.6twYy.6twYy.6xpaI.6xpaI.5-glM.5-glM.5-glM.6t_RI.6t_RI.6t_RI`

`counter.py`

`json_request.py`

`json_request.py`

`data-processing.py`

`data.py`

`graphs.py`