[PYTHON] Asynchronous http communication using asyncio

background

For example, suppose you want to continue detecting objects on an edge terminal and send the processing results to the server. If HTTP POST processing is inserted during the detection loop processing, the processing will stop until a response is returned. Asynchronous communication can be realized by using python standard async / await. I will summarize how to use it.

Reference site

• http://iuk.hateblo.jp/entry/2017/01/27/173449
• https://qiita.com/meznat/items/c34fad95dab593f9bffa
• https://qiita.com/icoxfog417/items/07cbf5110ca82629aca0
• https://note.crohaco.net/2019/python-asyncio/
• https://docs.aiohttp.org/en/stable/client_quickstart.html#json-request
• https://docs.python.org/ja/3/library/asyncio-future.html#asyncio.Future.add_done_callback

environment

• python3.7.3

Process flow

import asyncio
import aiohttp

def say_something(future):    # 5
    print(future.result())
    return 'hello!'


async def nested(counter):
    return counter


async def test():
    await asyncio.sleep(2)    # 3
    return'aaaaaaaaaaaaaaaaaaaa'


async def main():
    rep = 0
    counter = 0
    while True:

        if (counter % 100 == 0) and (counter != 0):
            http_req = asyncio.create_task(test())    # 2
            http_req.add_done_callback(say_something)    # 4
            counter = 0
            print(f'rep: {rep}')
            rep += 1

        task = asyncio.create_task(nested(counter))

        print(await task)

        counter += 1


asyncio.run(main())    # 1

What is async

Declaration for defining a coroutine. Coroutines are a more general form of subroutines that can have multiple entry points. That is, the process can be interrupted and restarted from that location.

What is an event loop?

An image like a task scheduler is fine. By registering a coroutine for the event loop, it automatically executes, stops, and reprocesses.

Contents of processing

1. Register the main coroutine in the event loop with asyncio.run.
2. Inside the main coroutine, register another coroutine test () in the event loop under specific conditions. The process of test () is started in parallel at the same time as registration.
3. Inside test (), await asyncio.sleep (2) performs a process called "wait for 2 seconds".
4. The processing of main () does not stop, but define the callback function to be executed when the currently registered test () = http_req ends.
5. The callback function displays the result of the future object (like the entity of http_req) on the standard output.

• The return value of asyncio.create_task () is a future object, which automatically registers the return value to itself when the process is completed. Can be called with .result ().

1. Register the process in the event loop and execute it even when it is not under specific conditions. Then the while loop continues.

Precautions for mounting

http requests did not use the requests library, but used a library called aiohttp.

$ pip install aiohttp

HTTP requests can be registered as coroutines in the following form.

async def process_http_request(data):
    async with aiohttp.ClientSession() as session:
        async with session.post(URL, json=data) as response:
            return await response.text()