I want to send barometric pressure information (obtained by Beautiful Soup) with a graph to Slack with Python (+ I want to manage it with Google Spread Sheet)

For the time being, Github

Source code (Github)

background

It seems that many people have the following symptoms when a low pressure system or a sudden drop in atmospheric pressure occurs.

--Physical: drowsiness, drowsiness, headache --Mental: Depressed or negative

Looking at my timeline, some people alert me when the air pressure drops. There are quite a few articles.

** I don't know if there is any scientific basis **

For the time being, I wanted to find out if my physical condition was related to atmospheric pressure.

Purpose

--To subjectively grasp your physical condition by looking at graphs and notifications of changes in atmospheric pressure. -Mathematically, is there a correlation by comparing with the record of mood changes? To confirm

Solution method ・ What to do this time

1. Get the latest barometric pressure <= This article
2. Slack notification of barometric pressure <= Next article
3. Save to Google Spread Sheet <= Next

(The tree command can also be used with win)

root/
│  .env
│  .env.sample
│  .gitignore
│  app.py
│  ---.json
│  README.md
│  requirements.txt
│
├─api_packages/
│  │  googleItem.py
│  │  slackItem.py
│  │  __init__.py
│
├─data_packages/
│  │  fetchHtml.py
│  │  generateGraph.py
│  │  __init__.py
│
├─imgs/

python=3.8

pip install beautifulsoup4
pip install requests
pip install numpy
pip install matplotlib
pip install python-dotenv
pip install gspread
pip install oauth2client

Get barometric pressure

I thought about getting it with API, but I decided to use the data of Tokyo published by the Japan Meteorological Agency. https://www.jma.go.jp/jp/amedas_h/today-44132.html

I just wanted to study Beautiful Soup, so I also do that.

fetchHtml.py

import requests
from bs4 import BeautifulSoup
import datetime

import re

class HtmlFetcher:
  url = None

  def __init__(self, url):
    self.url = url #I tried to make it variable, but stopped ...

  def fetch_pressure_from_jma(self, search_time = datetime.datetime.now()): #I'm taking the default argument, but I can't do this
    url = "URL you want to look up"
    try:
      res = requests.get(url)
      soup = BeautifulSoup(res.text, 'html.parser')
      trs = soup.select("div#div_table > table > tr")
      list_time_pressure = []
      for tr in trs:
        tds = tr.select("td")
        if tds[0].get_text().isdigit() and tds[8].get_text().replace('\xa0', ''):
          list_time_pressure.append(
            {
              'time': int(tds[0].get_text()),
              'pressure': tds[8].get_text()
            }
          )
      if len(list_time_pressure) == 0:
        return None
      else:
        date_place_title = soup.select("table#tbl_title td.td_title")[0].get_text()
        year = re.search(r'\d{4}Year', date_place_title).group()[0:4]
        month = re.search(r'\d{2}Month', date_place_title).group()[0:2]
        day = re.search(r'\d{2}Day', date_place_title).group()[0:2]
        place = re.search(r'\s+\D+\Z', date_place_title).group()[1:]
        result = {}

        # if search_time:
          #I wanted to get the data closest to the specified time ...
          # sorted_time_pressures = sorted(list_time_pressure, key=lambda x: abs(int(search_time.hour - x['time'])))
        #Get the most recent one
        sorted_time_pressures = sorted(list_time_pressure, key=lambda x:-x['time'])
        result = {
          'data': sorted_time_pressures,
          'info': {'day': day, 'month': month, 'year': year, 'place': place}
        }
        return result
    except Exception as e:
      print(e)
      return False

I think you can read it in an atmosphere somehow

trs = soup.select("div#div_table > table > tr")

If you are familiar with jQuery and SCSS, this is a rather intuitive notation. The above page had a simple structure, but on the contrary, there were not many Classs, so I had to do a deep acquisition method.

Quoted from the relevant page on March 01, 2020

if tds[0].get_text().isdigit() and tds[8].get_text().replace('\xa0', ''):

Here it is determined whether the line displays barometric pressure.

There is no barometric pressure data in the "time" and "hour" lines. So if the data can be converted to an integer, it seems that the atmospheric pressure is in the 9th column.

In addition, if the atmospheric pressure is not entered, it is unavoidable to obtain it, so the condition is added. I replaced it because it contains a mysterious character.

I was worried when I was writing an article about whether it was working properly here.

list_time_pressure = []

For example, if you get it at 1am, this list will look like 1 in length.

date_place_title = soup.select("table#tbl_title td.td_title")[0].get_text()

I am getting information about the time when the first place name and date were entered.

It seems that td_title is how to name the Class.

We will divide this using regular expressions.

#The first 4 digits of the integer is the year
#I can't get the data for 645, when the Taika Reform was renewed, and 20000 for the future, but ...
year = re.search(r'\d{4}Year', date_place_title).group()[0:4]
month = re.search(r'\d{2}Month', date_place_title).group()[0:2]
day = re.search(r'\d{2}Day', date_place_title).group()[0:2]
#The last is like a land name
place = re.search(r'\s+\D+\Z', date_place_title).group()[1:]

sorted(list_time_pressure, key=lambda x:-x['time'])

Sorted in order of closest time. I wondered if I could handle it if the date changed, but it seemed that the data on the previous day was not reset when the date changed, so I'm saying this.

Digression

I was tired here, so I decided to split the article.