I tried scraping Yahoo weather (Python edition)
Introduction
It's close to my own memo, but I tried it so I wrote an article. Originally I did a lot of research to do rasp pie signage, I didn't have the information I wanted to pinpoint, so I ended up investigating only the scraping method and managed to do it myself. (I wasn't a Python user or a person in the web area, so I struggled for a moment ...) Yahoo has a weather api, but I didn't use it because I only know the amount of precipitation.
C # edition https://qiita.com/MonoShobel/items/3ae0c051d2aadba698a4 There is.
References I referred to the following article when scraping with Python. [Scraping with python and letting Google Home talk about the weather forecast](https://xn--u9jwc776tibf.net/?p=1494) [Introduction to Python Web Scraping Practice](https://qiita.com/Azunyan1111/items/9b3d16428d2bcc7c9406)
There are some good and bad explanations about scraping itself, so you should read it.
By the way, it's Yahoo weather, but maybe scraping isn't prohibited. Yahoo Finance was explicitly banned, but I couldn't find any mention of the ban on the weather. Just keep moderation. Even if you scrape regularly, I think that access every hour or 30 minutes is enough for the weather.
What to do
I have a lot of information I want to get, but for the time being, I got the weather today in Tokyo.
Get today and tomorrow's weather in the image below.
Code
You can get the weather in Tokyo today and tomorrow by actually executing the code. (I wrote and debugged various things with Visual Code.)
Mostly in the comments you will find what you are doing.
It's almost a copy / paste source, but the point of interest is getting the text part with select_one.
I tried to get it automatically with Google Chrome, but it didn't work, so I searched by hand and it worked.
tenki.py
import urllib3
from bs4 import BeautifulSoup
#URL to access
url = 'https://weather.yahoo.co.jp/weather/jp/13/4410.html'
#Access the URL In the return value, the instance containing the access result and HTML etc. will be returned.
http = urllib3.PoolManager()
instance = http.request('GET', url)
#Extract HTML from instance and parse it for beautiful Soup
soup = BeautifulSoup(instance.data, 'html.parser')
#Get the weather text with CSS selectors.
#today's weather
tenki_today = soup.select_one('#main > div.forecastCity > table > tr > td > div > p.pict')
print ("How is the weather today"+tenki_today.text)
#Tomorrow's weather
tenki_tomorrow = soup.select_one('#main > div.forecastCity > table > tr > td + td > div > p.pict')
print ("Tomorrow's weather"+tenki_tomorrow.text)
#Wait for input so that the screen does not disappear in an instant
input()
tenki.py
import urllib3
from bs4 import BeautifulSoup
#URL to access
url = 'https://weather.yahoo.co.jp/weather/jp/13/4410.html'
#Access the URL In the return value, the instance containing the access result and HTML etc. will be returned.
http = urllib3.PoolManager()
instance = http.request('GET', url)
#Extract HTML from instance and parse it for beautiful Soup
soup = BeautifulSoup(instance.data, 'html.parser')
#Get the weather text with CSS selectors.
#today's weather
tenki_today = soup.select_one('#main > div.forecastCity > table > tr > td > div > p.pict')
print ("How is the weather today"+tenki_today.text)
#Tomorrow's weather
tenki_tomorrow = soup.select_one('#main > div.forecastCity > table > tr > td + td > div > p.pict')
print ("Tomorrow's weather"+tenki_tomorrow.text)
#Wait for input so that the screen does not disappear in an instant
input()
Below, the execution result
At the end
I think you can do various things based on this code.
I think that I will add it to this article or write it in the article of Razpa Signage if I get it regularly.
Also, I think that there are some things that could not be executed due to environmental problems, so I will add that later.
Recommended Posts
Recommended Posts