[PYTHON] Aggregate VIP values of Smash Bros. with Pandas

Introduction

[Scraping for the first time] I tried to make a VIP character for Smash Bros. [Beautiful Soup] [Data analysis]

Scraping

import re
import time

import requests
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko"
}

result = []

for i in range(1, 82):

    url = f"https://kumamate.net/data/?mode=rate&fighter={i}"

    r = requests.get(url, headers=headers)
    r.raise_for_status()

    soup = BeautifulSoup(r.content, "html.parser")

    for trs in soup.find_all("tr", class_=re.compile("RecentMatch[12]")):

        tds = trs.find_all("td")

        win, lose, _ = re.split("[Win or lose]", tds[3].get_text(strip=True))

        data = [
            tds[0].img.get("alt"),
            tds[2].get_text(strip=True),
            int(win),
            int(lose),
            float(tds[4].get_text(strip=True).rstrip("%")),
        ]

        result.append(data)

    time.sleep(1)

result

Data analysis

import pandas as pd

df = pd.DataFrame(result, columns=["myself", "Opponent", "Win", "Loss", "Win率"])

#Number of uses calculation
df["Number of uses"] = df["Win"] + df["Loss"]

#Total number of uses by fighter
pv = df.pivot_table(values="Number of uses", index="myself", aggfunc="sum").reset_index()

pv.rename(columns={"myself": "Opponent", "Number of uses": "Registration number"}, inplace=True)

df1 = pd.merge(df, pv, on="Opponent")

#VIP calculation
df1["VIP"] = df1["Registration number"] * df1["Win rate"]

#Exclude the same character
df2 = df1[df1["myself"] != df1["Opponent"]].copy()

#VIP aggregation by fighter
pv1 = df2.pivot_table(values="VIP", index="myself", aggfunc="sum").sort_values(
    by="VIP", ascending=False
)

pv1["VIP"] = (pv1["VIP"] // 100000).astype(int)

print(pv1.to_markdown())

Total number of uses

VIP tabulation

reference

[First scraping] I tried to make a VIP character for Smash Bros. [Beautiful Soup] [Data analysis]