python. Can programming beginners make things randomly? Part ①

This post is not a post where you can learn anything. It's just a post where beginners learn programming and write down where they stumbled. .. .. orz I'm good at python! This guy is worried about something simple. If you have a kind heart, please give me some advice. ..

environment: windows Anaconda3 python3.7.6

I would like to use python to export a list of videos containing a specific image from a youtube video thumbnail to a spreadsheet (Excel), such as a URL.

But there is a problem and I have never programmed at work. I've written VBA, and when I got a job, I learned the grammar of JAVA for about 3 months. And I don't have any friends to program. (Lol) Then do you give up? If you search with qiita, there are people who can put various programs that you can use immediately. It starts to move with a sweet expectation that it is quite so.

final goals

What I want to do in the end is an app that exports to a spreadsheet when the video contains the following "Goblin Barrel" from the channel "Stats Royal" which has many videos of the top players of the smartphone game "Clash Royale". Will be created. This game is a card game, and Stats Royal always uses thumbnails for battle decks, so image recognition? I wonder if you can search with. I think, but there is no confirmation. (Lol)

↓ Stats Royal thumbnail

↓ Goblin barrel

What you know

(1) If you use the youtube API, you can download thumbnails and get detailed channel data for free. └ It doesn't look like it's that difficult here. Isn't it? (2) I'm hoping that I'll study image recognition, face recognition, etc. and get closer to the deliverables, but it's probably different. └ I'm scared here. Maybe stray. ③ Spreadsheet operation was in the book "Let's let python do the boring things", so if you read it, it seems quite so. However, there is no image recognition. After all it depends on the net, qiita. ..

For the time being, move your hand even a little to get used to python

For python grammar, I read O'Reilly's "Let Python do the boring things" when taking a bath. The title is good, isn't it? The program does the "boring" thing! I yearn for it. (Laughs) Today I checked the flow control while. It's still a super introduction. However, I don't have the patience, so studying alone is not interesting. I searched with qiita and found an article containing the program, so I will move my hand first. I will understand it in detail later. I chose the following article because it contains a "program to download images from a website" and even an explanation is written, so it seems to be a learning experience. https://qiita.com/tigerz17/items/e4d1d5b8e00f7a771177 "Creating a slope group discrimination AI by image recognition"

Since I succeeded in executing "3. Extraction of face", I will write a memorandum of the stumbled part.

The part that stumbled when I moved my hand

First of all, the execution environment of python called ANACONDA? Installed on windows.

After that, copy and paste the program of "2. Image collection" without changing any characters (laughs). Of course it didn't work. Error (1) Import is not possible in the first place. cv2, webdriver Error (2) blog_list = driver.find_elements_by_class_name ('blog') class called blog has an error Error ③ Login authentication was requested at the time of go to next page

Regarding error (1): These are like the libraries used for image recognition, but pip install was required to import them. Also, I'm not sure about one point, but it's not in the program on the above site.

import chromedriver_binary

It went well when I put in. It's important to move forward, thinking that you have to know why it works if you put it in and if it doesn't. Start walking without looking back. .. .. (Maybe per environment variable.)

Regarding error (2): This was the hardest. I didn't know where the error was. The blog class is suspicious because I ran it line by line in IDLE (interactive mode shell). I was able to reach the question. Yes, you also have to know HTML / CSS. That's right. .. .. orz For the time being, when I right-clicked the image in chrome → pressed verification, various class names appeared nearby, and if I changed the blog to images, it worked. I'm moving forward without understanding it at all. (Does this mean that the management changed the class name? Is it a countermeasure because everyone uses this program and the access load is heavy?)

Regarding error ③: When the program downloads all the images on the first page, it goes to the second page to search for the images. At that time, you will be asked to log in to twitter. The program tries to go to the next immediately, so an error is returned. Putting time.sleep (20) and completing the login process within 20 seconds is solved by brute force without understanding again. (Lol) I would like to confirm the following this weekend. (I wonder if it matches) https://qiita.com/syunyo/items/36af8bcb501baf8c7014 "[Python3] Scraping on a site with login function [requests] [Beautiful Soup]"

Yes. As you can guess, we haven't captured it in a straightforward manner. I managed to do it with brute force. It will be somehow. I am writing this article so that I can look back and learn properly later. (Laughs) Kuu

I was able to avoid the error safely and got 3500 images of Nogizaka. (The feeling of leaving the computer waiting for the image to be downloaded was very fresh and happy. (Laughs) It feels like programming.)

Next is "3. Extraction of face". Copy the program again. Of course it is an error. (Lol) However, there is only one error. When I ran it line by line with IDLE, the following was a problem.

#Specify the path of the cascade file
    cascade_file = "C:/haarcascades/haarcascade_frontalface_alt.xml"

Ah, the file save destination is different. That's right.

import cv2
print(cv2)

If you execute, you can find out your environment information, and if there is "haarcascade_frontalface_default.xml" nearby, that's it. .. It was in my environment below.

C:\ProgramData\Anaconda3\lib\site-packages\cv2\data\haarcascade_frontalface_default.xml

Put this in the variable above. .. .. It worked. For the time being, I'll see who this guy is again. (I will postpone my favorite study again.) Now that the program has started, I can't keep up with it. Thank you for reading. orz ↓ This image is ...

↓ This is what happened. (There are many beautiful women in Nogizaka. I don't know.)

PS: Face recognition doesn't seem to be very accurate at all, but it's like this. .. .. ?? There are many no faces, and there are images of skirts that have nothing to do with the face. Hmm.