What I made

It is a LINE BOT that notifies you when the entered time (seconds) has passed. It's not interesting, but it was a good subject for asynchronous processing. Below is a screenshot of what I created, but if you type "180" you will be notified after 180 seconds. Of course, other messages need to be processed during that 180 seconds.

However, due to the specifications of LINE's API, requests from users time out within 180 seconds, making it impossible to reply. It would be nice if you could reply before the timeout in the first place, When a large amount of processing comes, heavy processing such as image processing that can not be made in time, Something like this timer can only be processed after returning the request once.

Basic policy

For the time being, the LINE BOT server runs on heroku like many others do. SSL communication is essential for LINE BOT, so it is a little troublesome to prepare by yourself, This is possible by default on heroku. That's the advantage of PaaS.

I referred to the following article about the architecture of LINE BOT.

Articles about LINE BOT 1.1. LINE BOT server architecture that is safe even if a large number of messages come
Articles about heroku 2.1 Worker Dynos, Background Jobs and Queueing 2.2 Background Tasks in Python with RQ

Background workers for asynchronous processing use RQ according to 2.2. You need a Redis server to use Race Queen, but heroku is easy because you only need to add an add-on called RedisToGo.

$ heroku addons:create redistogo

When doing LINE BOT with Race Queen, if you roughly draw the MessageEvent when a message is received from the user, the sequence will be as shown in the figure below.

Requests (sender ID, message, reply token) arrive from the LINE server to the BOT server
Reply a message to the LINE API server using the reply token (not required)
Register the job for the message in RQ (Job Queue)
Return 200 to the LINE server
Remove the job from the job queue
Process the job
Notify the LINE API server of the job results

Here, the message sent to the LINE API server is displayed on the user's LINE screen. The important thing is that the time from 1 to 4 is limited both in terms of LINE and heroku. By the way, the reply token also has an expiration date. If you use it, let's use it quickly. And immediately return 200 to the LINE server. If you know the sender ID, you can run 5-7 at any time. (It's a problem that the result notification is too late)

The following three processes exist in this LINE BOT. I can't create a timer thread for each request, so I decided to manage the timer with Redis.

LINE BOT server: Accepts requests and registers jobs in RQ
RQ Worker: Take out a job and register a timer in the sorted set of Redis (sort by set time)
Timer worker: Regularly monitors Redis, notifies the sender if there is a timer that has reached the set time, and deletes the timer

The Timer worker doesn't appear in the sequence above, Since the Race Queen worker is executed only when the job is registered, the timing of the timer is unknown. Therefore, the Timer worker detects whether the timer has reached the set time and notifies the LINE API server.

Heroku free tier restriction measures

Source code of this time, but it will not work with heroku's free frame as it is. I have specified one web server and two background workers in the Procfile, This is because you can only set two dynos at a time in the free tier.

`Procfile`


web: gunicorn app:app
rq_worker: python rq_worker.py
timer_worker: python timer_worker.py

$ heroku ps:scale rq_worker=1 timer_worker=1
Scaling dynos... !
 !    Cannot run more than 2 Free size dynos.

For this reason, unfortunately, I took the method of modifying the same source code only in Procfile and deploying it as two apps.

`Procfile for LINE BOT`


web: gunicorn app:app
rq_worker: python rq_worker.py

`Procfile for timer`


timer_worker: python timer_worker.py

In this case, the two apps must connect to the same Redis server, so you need to set the environment variable REDISTOGO_URL of the RedisToGo add-on created by one app to the environment variable of the other app.

bonus

Since the LINE BOT I made this time does not process much for input, an error will occur except for numbers.

By the way, I also made a LINE BOT that morphologically analyzes the input and makes a mess (source code: github).

Asynchronous processing using LINE BOT: RQ (Redis Queue) in Python