[PYTHON] [AWS] Judge the existence of Next Token and take all the values [boto3]

Good morning Anderson This is my memorandum. Save it

The beginning of the matter

There was a script on AWS Lambda that digs with EC2 instance information (describe_instances). I noticed that I didn't get all the instanceIDs. Yes, boto3 has an API called NextToken, which has an upper limit for acquisition at one time.

what is that

Please read the boto3 docs ... it won't do anything ... See here as an example https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_instances

EC2 describe_instances API Return Response Syntax

'string'

 There should be a value.

 How many values can be taken at one time? I'm curious
 ** Maybe about 50? ** Maybe it depends on the API. If the value of MaxResults is the upper limit, describe_instances will be up to 1000.
 Anyway, the key such as Next Token is the symbol of "Get it from here next".

 For the time being, I've gotten a script that can take all the cases.
 Well, ask support or read the documentation for details (terrible)

 what! ?? This side-dressing thing! Give me the result! ??
 Recognize.

# Script sample example
```python
NextToken = None
while True:
    if NextToken is None:
        ec2_var = ec2_cli.describe_instances()
    else:
        ec2_var = ec2_cli.describe_instances(NextToken=NextToken)
    
    #Get the required value from the EC2 return value (JSON).
    for Reservations in ec2_var['Reservations']:
        for instances in Reservations['Instances']:
            #Host name (tag/Name) Extraction
            ec2_list_nametag.append(instances['Tags'][0]['Value'])
            #Host instance ID extraction
            ec2_list_id.append(str(instances['InstanceId']))
            #Host placement AZ information extraction
            ec2_list_az.append(instances['Placement']['AvailabilityZone'])
            #Host status extraction
            ec2_list_status.append(instances['State']['Name'])
    
    if not 'NextToken' in ec2_var:
        break
    NextToken = ec2_var['NextToken']
else:
    print(NextToken)

#Combine the values from the extracted EC2 list
ec2_list = [ec2_list_nametag,ec2_list_id,ec2_list_az,ec2_list_status]

#(Example) AZ-Get the host you want to be in A.
for i in ec2_list[0]:
    if 'hoge' in i:
        num =ec2_list[0].index(i)
        if ec2_list[2][num] == 'ap-northeast-1a':
            hostname = str(i)

#(Example) Checking the status of your destination.
if hostname in != '': #Let's do it if you are looking for it
    for i in ec2_list[0]:
        if hostname in i:
            num =ec2_list[0].index(i)
                host_status = str(ec2_list[3][num])
else:
    print('Doesn't it look like you didn't have an address?')

result

By using the above script, I was able to store all instance information in a variable. Then, dig up the required values in the internal for and make a list. Due to the ridiculous toys such as EMR, even if EC2 is started a few times, it will be able to be taken quickly.

Supplement

I'm using a multidimensional array, but by using this, I can dig various things from the whole amount of one. What do you want to say? If you hit the API and are in a good mood, throttling will occur and processing will be slow. If you want to add something like a retry process, it's easier to take it first with one shot, isn't it? That is the theory. Wow this So, if you trace the same return value with for from the top, the order will be the same, right? That is the theory. Wow this Is it easy to see that I didn't use the intensional notation? It's my own concern that anyone can understand it. You may be asked to consider the memory usage of variables, but if you run it on AWS Lambda, you should be a little concerned. Amount of memory.

That's it.