There is a library called huey that implements task queues in python. Compared to celery, which is famous for the library of the same task queue, you can implement periodic job execution more simply. This time, I will not touch on normal task execution.
--Installing the library
pip install huey
pip install redis
--Starting Redis for broker
Easy to start with docker
docker run --rm -d -p 6379:6379 redis
This time we started Redis on the broker, so we call Redis Huey. You can register a periodic job just by adding a periodic_task decorator to the function.
main.py
from huey import RedisHuey, crontab
import time
huey = RedisHuey(host="127.0.0.1")
@huey.periodic_task(crontab(minute="*"))
def test():
print("process start")
time.sleep(5)
print("process end")
Specify the job execution time with the crontab given as the decorator argument. This time is every minute.
After installing huey, you can use the command huey_consumer, which will start the worker process.
$ huey_consumer main.huey
[2020-07-16 23:18:59,827] INFO:huey.consumer:MainThread:Huey consumer started with 1 thread, PID 2744 at 2020-07-16 14:18:59.827805
[2020-07-16 23:18:59,828] INFO:huey.consumer:MainThread:Scheduler runs every 1 second(s).
[2020-07-16 23:18:59,828] INFO:huey.consumer:MainThread:Periodic tasks are enabled.
[2020-07-16 23:18:59,828] INFO:huey.consumer:MainThread:The following commands are available:
+ main.test
[2020-07-16 23:18:59,836] INFO:huey.consumer.Scheduler:Scheduler:Enqueueing periodic task main.test: 30a61be7-903c-4fa7-815f-c5c013074085.
[2020-07-16 23:18:59,841] INFO:huey:Worker-1:Executing main.test: 30a61be7-903c-4fa7-815f-c5c013074085
process start
process end
[2020-07-16 23:19:04,847] INFO:huey:Worker-1:main.test: 30a61be7-903c-4fa7-815f-c5c013074085 executed in 5.005s
[2020-07-16 23:19:59,830] INFO:huey.consumer.Scheduler:Scheduler:Enqueueing periodic task main.test: 8166b273-1b97-427a-a349-a2772ea67fd2.
[2020-07-16 23:19:59,834] INFO:huey:Worker-1:Executing main.test: 8166b273-1b97-427a-a349-a2772ea67fd2
process start
process end
[2020-07-16 23:20:04,839] INFO:huey:Worker-1:main.test: 8166b273-1b97-427a-a349-a2772ea67fd2 executed in 5.005s
You can see that the job is running every minute.
that's all
Recommended Posts