[PYTHON] Use the retry processing mode added to Boto3

Introduction

The AWS SDK implements automatic retry logic with Exponential Backoff, Boto3 offers three retry processing modes.

• legacy (default)
• standard
• adaptive

Checking botocore's GitHub repository, standard and adaptive are found around February 2020. Added and earlier implementations are named legacy.

Add support for new retry modes #1972

For compatibility, the default is legacy, and features added in standard / adaptive To use it, you need to explicitly change the retry processing mode.

How to change the retry processing mode

Instantiate the botocore config object and send the configuration information to the client You can pass it. The available options are max_attempts and mode. The default maximum number of retries varies depending on the retry processing mode, Can be customized by max_attempts.

import boto3
from botocore.config import Config

config = Config(
   retries = {
      'max_attempts': 10,
      'mode': 'standard'
   }
)

ec2 = boto3.client('ec2', config=config)

Overview of each retry processing mode

Legacy retry mode This is the default mode used by Boto3 clients. The factor used to calculate the backoff time is 2. The v1 retry handler (https://github.com/boto/botocore/blob/master/botocore/retryhandler.py) is used. There are a limited number of errors / exceptions for retries.

Standard retry mode Retry rules are standardized with multilingual AWS SDKs and consistent logic. It handles more throttling / limit errors and exceptions than legacy mode. The factor used to calculate the backoff time is 2, as in legacy mode, but with a maximum of 20 seconds. The v2 retry handler (https://github.com/boto/botocore/tree/master/botocore/retries) is used.

Adaptive retry mode In addition to the standard mode functionality, the token bucket algorithm is used on the client side. Automatic rate limiting feature and AWS service side error / exception / HTTP status code for each retry Added the ability to change rate limiting variables based on. Flexible re-execution can be performed on the client side according to the error content, ** This is an experimental mode, so its functionality and behavior may change in the future. ** **

reference

Retries - Boto3 Docs https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html

that's all. I'm glad if you can use it as a reference.