[PYTHON] Posted the number of new corona positives in Tokyo to Slack (deployed on Heroku)

In [Previous article] 1, we introduced a bot that posts [Number of people positive for new coronavirus in Tokyo] 2 to Slack.

This article is the sequel to that.

Additional development points from the previous time

• Automatically post to Slack after data is updated
• [Last time] 1 is regularly executed by Crontab
• Deploy on Heroku
• [Last time] 1 run on home machine

As shown below, it was automatically posted after today's data update.

Development 1: Automatically post to Slack after data is updated

[Page where Tokyo publishes data] I checked the csv data obtained from 2 and implemented it so that if there is a difference, I will post it.

[Last time] 1 Let's call the created method to post to Slack fdap.main (). Then, prepare fdap.main () as run2.py as the source code to be executed after [Tokyo data] 2 is updated.

The contents of run2.py.

run2.py

import run as fdap
import post
import plot
import time
import datetime

def check_data(last_no):
    csv_tail_no = plot.fetch_csv().iat[-1,0] # "No"Get the last row of a column
    if int(last_no) == 0:
        return False, csv_tail_no # initialize
    elif int(csv_tail_no) > int(last_no):
        return True, csv_tail_no
    else:
        return False, csv_tail_no

def main():
    while True:
        if "csv_tail_no" in locals(): #After the second loop
            updated, csv_tail_no = check_data(csv_tail_no)
        else: #1st loop
            updated, csv_tail_no = check_data(0)
        if updated:
            fdap.main() #Post to Slack
        time.sleep(60*5) #5 minutes

if __name__ == "__main__":
    main()

In check_data (last_no), check the No column in the last row of [Tokyo data] 2, and if there is a difference from the previous confirmation, describe the process so that it will be posted to Slack. doing.

Regarding the difference judgment method, this time we are using this method for the sake of ease of implementation, but it may be smarter to scrape the modification date and time of the file. However, I saw [the page where the data of Tokyo is published] 2 several times, but it seemed that the last update date and time did not match the data update timing, so scraping the update date and time. I felt that I needed to take other measures.

It is described that the data judgment is executed at regular intervals. In order not to access the data more than necessary, I put a sleep for 5 minutes this time.

Development 2: Deploy on Heroku

Since python3 run2.py is always executed, prepare the execution environment on Heroku.

Prepare the necessary files

Prepare runtime.txt, requirement.txt, Procfile.

In runtime.txt, describe the version when deploying with Heroku.

runtime.txt

python-3.6.10

Describe the required modules in requirements.txt. Created with pip freeze> requirements.txt.

(I manually deleted the modules that seemed to be unnecessary, but they may still be there ...)

requirements.txt

jsonpatch==1.10
jsonpointer==1.9
python-dotenv==0.12.0
requests==2.9.1
slackbot==0.5.6
matplotlib==3.0.3
numpy==1.18.1
pandas==0.24.2
scipy==1.4.1

Procfile describes the process name and the command you want to execute.

Procfile

c19bot: python3 run2.py

Setting

Create a Heroku account in advance.

Sign in to Heroku from your browser and create a new application with an appropriate app name (select New-> Create new app at the top right of the screen).

On the Setting tab, under Build packs, select Python. Also, enter the required environment variables (Slack Token, channel name, etc.) in Config Vars.

The rest of the work is done with the Heroku CLI. After logging in with Heroku login and reflecting the source code remotely, deployment is completed with heroku scale [process name] = 1.

Things to watch out for (environment variables)

When developing locally, environment variables are implemented to be read from the .env file.

Below is an example of the source code.

from dotenv import load_dotenv
load_dotenv(dotenv_path=".env") # load local .env
import os

SLACK_TOKEN = os.environ["SLACK_TOKEN"] 
USER_NAME = os.environ["USER_NAME"]
SLACK_CHANNEL = os.environ["SLACK_CHANNEL"]
CHANNEL_ID = os.environ["CHANNEL_ID"]
PNG = os.environ["PNG"]
PNG_FILE = PNG+".png "

This time, you need to save the image (graph of the number of positive people) posted to Slack as a temporary file. If the image name is hist.png, you can save it in the current directory like PNG = "./hist" when developing locally, but when deploying with Heroku, name the file Setting it to "/ tmp / hist"(save it under / tmp) works fine. Therefore, when developing locally, the file name of the image is also added to the environment variable in advance, and the file name is specified separately on Heroku.

Things to watch out for (Dyno's life cycle)

Heroku's process seems to restart automatically on a regular basis.

Reference: https://jp.heroku.com/dynos/lifecycle (Heroku Dyno Lifecycle)

Initially, I wrote a program to post the latest information to Slack once when python3 run2.py was started. However, about a day after deploying on Heroku, there was a bug that somehow [Tokyo data] 2 was posted to Slack even when it was not updated.

After some research, I realized that the Heroku process might have been restarted automatically, and fixed the program. Currently, when python3 run2.py is started, it is not posted to Slack, but the data is posted to Slack for the first time when the data in Tokyo is updated after starting.

Summary

[Previous article] Continuing from 1, we automated the bot that posts the number of people positive for the new coronavirus in Tokyo to Slack and deployed it on Heroku (resident).

The information posted to Slack should be taken as supplementary information only, and I would like to keep in mind to check [Latest infection trends in Tokyo published on the official website of Tokyo] 3 once a day. I will.

that's all.