[PYTHON] Celery asynchronous processing in Flask

Introduction

I was writing a simple python application in Flask. So I wanted to add asynchronous processing. However, there were some points that I got stuck in, so I wrote an article. One thing I can say first is that asynchronous processing in python may be better with a non-Flask framework. When I was googled, I felt that other frameworks had more information.

Execution content

The user types two numbers into the text box and presses a button. The server side performs the operation of adding the two numbers asynchronously and returns the response to the user immediately.

Execution environment

os: Windows10 redis Redis-x64-3.0.504.msi

python 3.5.6 (Anaconda) Flask==1.1.1 celery==3.1.25 redis==2.10.6

Execution procedure

--Run "python sample.py" in Anaconda Prompt. --Run "celery -A sample.celery worker -l info" in Anaconda Prompt. --Access localhost: 9000, type a number in the text box and press Execute Calculation. view.png

--As a result, a response is returned immediately. Confirm that the calculation result is displayed in Anaconda Prompt that executed celery. view2.png

Stumble point

--Since the versions of celery and redis are compatible, first check these.

--Check if redis is running on localhost: 6397.

--Check if the function to be processed asynchronously is registered in the task of celery. (Can be confirmed on the terminal where "celery -A sample.celery worker -l info" is executed)

in conclusion

It's easy, but I've attached sample.py and main.html below. It works with these two files. I also attached the URL of the site I referred to.

If you have any questions or mistakes, please comment.

Source code

sample.py


# coding=utf-8
from flask import Flask, render_template, request

#Flask app preparation
app = Flask("sample")

# this is a part of celery property to use in Flask
from celery import Celery
def make_celery(app):
    celery = Celery(
        app.import_name,
        backend=app.config['CELERY_RESULT_BACKEND'],
        broker=app.config['CELERY_BROKER_URL']
    )
    celery.conf.update(app.config)

    class ContextTask(celery.Task):
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery

# update flask config to use celery
app.config.update(
    CELERY_BROKER_URL='redis://localhost:6379',
    CELERY_RESULT_BACKEND='redis://localhost:6379'
)
celery = make_celery(app)

@celery.task()
def add(a, b):
    print(a + b)
    return a + b

@app.route('/', methods=["GET", "POST"])
def sample():

    #Processing at POST
    if request.method == 'POST':
        return render_template(
            'main.html',  #HTML file to display
            )

    #Processing at the time of GET
    else:
        return render_template(
            'main.html',  #HTML file to display
            )

@app.route('/celery', methods=["POST"])
def celery_test():

    if request.method == 'POST':
        a = int(request.form["num_one"])
        b = int(request.form["num_two"])
        result = add.delay(a,b)
        return render_template(
                'main.html',  #HTML file to display
                )

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=9000)

main.html


<!DOCTYPE html>
<html lang="ja">
  <head>
    <meta charset="utf-8"/>
    <title>Sample</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
  </head>
 
  <body>      
    <div class="container">
      <div class="jumbotron">
        <h1>
          <p align="center">
            <font size="10">Flask asynchronous processing</font>
          </p>
        </h1>
      </div>

      <form class="" method="post" action="/celery" enctype="multipart/form-data">
        <div class="container" style="text-align: center;">
          <input type="number" name="num_one">  +  
          <input type="number" name="num_two">
        </div>

        <div class="button_wrapper" style="text-align: center;">
          <button type="submit" class="btn btn-lg btn-success" style="margin-top: 50px; ">Calculation execution</button>
        </div>

      </form>
    </div> 

  </body>
</html>

References

Install Redis for Windows and try it out http://kageura.hatenadiary.jp Flask official https://flask.palletsprojects.com/en/1.0.x/patterns/celery/ python – Unable to launch Celery Worker https://codeday.me/jp/qa/20190525/900758.html celery AttributeError: 'str' object has no attribute 'items' django Day14 http://www.programmersought.com/article/8120463251/

Recommended Posts

Celery asynchronous processing in Flask
Asynchronous processing implementation in Django (Celery, Redis)
Asynchronous processing (threading) in python
About parameter processing in Flask request.args
Asynchronous processing using Linebot in Job queue
Multithreaded processing in python
Text processing in Python
Queue processing in Python
Image uploader in Flask
HTTP environment variables in Flask
Image Processing Collection in Python
Asynchronous processing using LINE BOT: RQ (Redis Queue) in Python
Using Python mode in Processing
Upload multiple files in Flask
Flow of getting the result of asynchronous processing using Django and Celery
Processing order when chaining when in PySpark
Signal processing in Python (1): Fourier transform
Create a CSV reader in Flask
100 Language Processing Knock Chapter 1 in Python
Paper: Music processing in the brain
Fixed Apscheduler running twice in Flask
Batch processing notes in IPython Notebook
Configuration file best practices in Flask
I tried asynchronous processing using asyncio