[PYTHON] I tried to search videos using Youtube Data API (beginner)

Introduction

Input is important for studying data analysis, but practice is the best, so I thought that there would be good data to practice. To be honest, I can't judge whether the Youtube data is good. However, I often watch Youtube, and since it is an area of interest, I would like to summarize how to use it with the goal of being able to extract data for analysis using ** "Youtube Data API" **. I used the following page (API reference) to learn the API. https://developers.google.com/youtube/v3/docs?hl=ja

Search process

This time, as a starting point, search the video under the following conditions and output the result to a csv file.

--Search videos with specified keywords (keywords are specified with the first argument) --Search results are displayed in descending order by the number of views

In addition, the video of the search result is frequency-distributed to which channel it belongs to and output to a csv file.

Source code

The source is as follows. For the variable "DEVELOPER_KEY" in the program, enter your own API key. The method of issuing the API key is omitted here.

searchKeyword.py

# import library
from apiclient.discovery import build
from apiclient.errors import HttpError
import argparse
import numpy as np
import pandas as pd

# Set Yotube Data API key
DEVELOPER_KEY = "YOUR API　KEY!!!"
YOUTUBE_API_SERVICE_NAME = "youtube"
YOUTUBE_API_VERSION = "v3"

def searchKeyword(options):
    #Keyword search process
    youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, 
                    developerKey=DEVELOPER_KEY)
    searchResults = youtube.search().list(q=options.sw,
                                        type="video",
                                        part="id,snippet",
                                        maxResults=options.max_results,
                                        order="viewCount"
                                        ).execute()
    
    #Search result classification processing
    videos = []
    others = []
    for searchResult in searchResults["items"]:
        if (searchResult["id"]["kind"] == "youtube#video"):
            videos.append(searchResult)
        else :
            others.append(searchResult)

    #Video, channel information formatting, csv file output
    videoTitles = []
    viewCounts = []
    likeCounts = []
    dislikeCounts = []
    favoriteCounts = []
    commentCounts =[]
    videoChannelTitles = []
    stat_list = [viewCounts, likeCounts, dislikeCounts, favoriteCounts, commentCounts]
    stat_keywords = ['viewCount', 'likeCount', 'dislikeCount', 'favoriteCount', 'commentCount']
    for video in videos:
        videoDetail = youtube.videos().list( part="statistics, snippet",
                                            id = video["id"]["videoId"]
                                            ).execute()
        channelDetail = youtube.channels().list(part="snippet", 
                                                id=videoDetail["items"][0]["snippet"]["channelId"]
                                                ).execute()
        
        videoTitles.append(videoDetail["items"][0]["snippet"]["title"])
        for stat, stat_keyword in zip(stat_list, stat_keywords):
            try:
                stat.append(videoDetail["items"][0]["statistics"][stat_keyword])
            except KeyError:
                stat.append(0)
        videoChannelTitles.append(channelDetail["items"][0]["snippet"]["title"])

    df_videos = pd.DataFrame({"title":videoTitles, "ViewCount":viewCounts, 
                            "channelTitle":videoChannelTitles,"likeCount":likeCounts,
                            "dislikeCount":dislikeCounts, "favoriteCount":favoriteCounts,
                            "commentCount":commentCounts})
    df_videos.to_csv("Search_result_{}.csv".format(options.sw),encoding="utf-8_sig")
    df_videos_countbyChannel = df_videos["channelTitle"].value_counts()
    df_videos_countbyChannel.to_csv("ChannelTitle_{}.csv".format(options.sw),encoding="utf-8_sig")

    return df_videos, df_videos_countbyChannel
    


if __name__ == "__main__":
    # parse Argument
    parser = argparse.ArgumentParser("search Youtube Program...")
    parser.add_argument("sw", help="search Keyword in Youtube")
    parser.add_argument("--max_results", type=int, help="max of search results",
                        default=50)
    options = parser.parse_args()

    searchKeywordResults = searchKeyword(options)

I tried to run

I actually moved it. This time, specify "quantum computer" as the search keyword and execute.

$ python searchKeyword.py "Quantum computer"

"Search_result_quantum computer.csv" and "ChannelTitle_quantum computer.csv" are created in the directory where "searchKeyword.py" is placed. Let's check the contents of these two files.

--Search_result_ Quantum computer.csv (only the beginning part is described)

It seems that I was able to get the video information well.

--ChannelTitle_Quantum computer.csv (only the beginning part is listed)

It seems that I was able to get the information well here.

Finally

If you apply this, you can do various interesting things. I will expand the functions little by little so that I can do a little more.