[PYTHON] A verification of AWS SDK performance by language

background

About tags

AWS can add tags to resources. Tags in AWS are used for resource management and cost allocation. Grant design is a very important point in AWS operation. Especially in a large environment or organization, the number of tags attached to resources tends to be very large.

AWS SDK for Ruby Performance

As mentioned in this article, AWS SDK for Ruby gets a lot of resources at once when a lot of tags are attached to the resources. May take a significant amount of time. The following is a graph comparing how long it took to complete resource acquisition (describe_volumes) with and without tagging, using EBS as an example (the image below is a reprint of the above article). If you have a large number of tags (20 in this example), you can see that it takes a very long time to complete the resource acquisition.

This article estimated that it was the part that took the time to parse the XML of the API response to the Ruby object. ..

Question

I was wondering if the above performance changes would occur in other languages of the AWS SDK, so I decided to check it.

Verification

This time, in addition to Ruby, I measured the performance with the AWS SDKs of Python, PHP, Go, and Node.js. The target is ** 1000 EBSs with 20 tags **. The client is LightSail (Ubuntu 18.04) in the Tokyo region. It is carried out 3 times in each language and the average value is taken. Each language and SDK version will be described later.

inspection result

The verification results were as follows. Apparently ** only the Ruby SDK ** took a lot of time. For other languages, I'm glad I don't have any concerns about this.

Conclusion / Impressions

It seems that only the AWS SDK for Ruby takes a lot of time to process with a configuration like this. As for the Ruby SDK, the performance may be improved by improving the library used internally, so I would like to expect it in the future.

Version and verification code of each language and SDK

Ruby

require 'aws-sdk-ec2'

Aws.config.update(credentials: Aws::Credentials.new('',''))
ec2_client = Aws::EC2::Client.new(region: 'ap-northeast-1')

start_time = Time.now

ec2_client.describe_volumes

end_time = Time.now
colapsed_time = end_time - start_time
puts colapsed_time

PHP

<?php
require 'vendor/autoload.php';

use Aws\Ec2\Ec2Client;

$ec2Client = new Aws\Ec2\Ec2Client([
  'region' => 'ap-northeast-1',
  'version' => '2016-11-15',
]);

$time_start = microtime(true);

$result = $ec2Client->describeVolumes();

$time = microtime(true) - $time_start;
echo "{$time}Seconds";

Python

import boto3
import time

ec2 = boto3.client('ec2')
start = time.time()

ec2.describe_volumes()

elapsed_time = time.time() - start
print ("elapsed_time:{0}".format(elapsed_time))

Node.js

var AWS = require('aws-sdk');

AWS.config.update({region: 'ap-northeast-1'});

var ec2 = new AWS.EC2();
const startTime = Date.now(); //Start time
ec2.describeVolumes({}, function (err, data) {
  console.log(Date.now() - startTime);
});

Go

package main

import (
	"fmt"
	"time"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/ec2"
)

func main() {
    sess, _ := session.NewSession(&aws.Config{
        Region: aws.String("ap-northeast-1")},
    )

    svc := ec2.New(sess)
    input := &ec2.DescribeVolumesInput{}

    start := time.Now()
    svc.DescribeVolumes(input)
    end := time.Now()
    fmt.Printf("%f seconds\n", (end.Sub(start)).Seconds())
}