I created a template creation tool for CloudFormation using troposphere. Also, by running the created tool on Lambda, I tried to create a template just by placing the parameter sheet on S3.
Recently, I had the opportunity to touch CloudFormation in my business. I think the bottleneck in using CloudFormation is the JSON format file.
python
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "hogehoge",
"Resources": {
"test1": {
"Properties": {
"CidrBlock": "10.1.0.0/16",
"EnableDnsHostnames": "true",
"EnableDnsSupport": "true",
"Tags": [
{
"Key": "Name",
"Value": "test1"
}
]
},
"Type": "AWS::EC2::VPC"
},
"test2": {
"Properties": {
"CidrBlock": "10.2.0.0/16",
"EnableDnsHostnames": "true",
"EnableDnsSupport": "true",
"Tags": [
{
"Key": "Name",
"Value": "test2"
}
]
},
"Type": "AWS::EC2::VPC"
}
}
}
It's like this just to make two VPCs, so I think it's quite a craftsmanship to write this by hand.
troposphere is a tool to create such a JSON format file by writing it in Python. https://github.com/cloudtools/troposphere If the above code is also written using troposphere, it can be written in an easy-to-read manner as follows.
python
import json
from troposphere import Tags,Template
from troposphere.ec2 import VPC
t = Template()
t.add_version("2010-09-09")
t.add_description("hogehoge")
t.add_resource(VPC(
"test1",
EnableDnsSupport="true",
CidrBlock="10.1.0.0/16",
EnableDnsHostnames="true",
Tags=Tags(
Name="test1"
)
))
t.add_resource(VPC(
"test1",
EnableDnsSupport="true",
CidrBlock="10.2.0.0/16",
EnableDnsHostnames="true",
Tags=Tags(
Name="test2"
)
))
json_template = t.to_json()
print(json_template)
The basic usage is to describe the version name and description with add_version and add_description, and then add more resources to be created with add_resource. It's not dramatically easier to see, but I think it's a little better than writing it in JSON format. Also, not only is it easier to see, but the point is that you can use Python for and if statements. It is possible to easily describe a large amount of resources by describing by passing multiple CIDR blocks and Names as an array.
python
import json
from troposphere import Tags,Template
from troposphere.ec2 import VPC
t = Template()
t.add_version("2010-09-09")
t.add_description("hogehoge")
VPC_CidrBlockList = ["10.1.0.0/16","10.2.0.0/16"]
VPC_TagsNameList = ["test1","test2"]
for (Address,Tag_Name) in zip(VPC_CidrBlockList,VPC_TagNameList):
t.add_resource(VPC(
Tag_Name,
EnableDnsSupport="true",
CidrBlock=Address,
EnableDnsHostnames="true",
Tags=Tags(
Name=Tag_Name
)
))
json_template = t.to_json()
print(json_template)
By the way, in the above state, setting values such as CIDR block and Name will be written in a script, but this is difficult to use. Let's read the parameters from the outside and store them in an array so that they can be used. In this case, there are two types of information required as parameters (CIDR block, Name), so create a parameter sheet as shown below and save it as a text file. 1st line 10.1.0.0/16, 10.2.0.0/16 2nd line test1, test2
After creating a text file, first open and read this text file.
python
f = open('test.txt')
test = f.read()
f.close
The data is now stored in test. However, in this state, the first line and the second line are not separated. So, use splitlines to store each row as an array.
python
test_list= test.splitlines()
This will create an array test_list with each row as an element. ["10.1.0.0/16,10.2.0.0/16","test1,test2"] Furthermore, here, split is used to create an array of each parameter.
python
VPC_CidrBlockList = test_list[0].split(',')
VPC_TagNameList = test_list[1].split(',')
Now you have created an array of parameters. If you incorporate these operations and modify the script above, it will be as follows.
python
import json
from troposphere import Tags,Template
from troposphere.ec2 import VPC
f = open('test.txt')
test = f.read()
f.close
test_list= test.splitlines()
VPC_CidrBlockList = test_list[0].split(',')
VPC_TagNameList = test_list[1].split(',')
t = Template()
t.add_version("2010-09-09")
t.add_description("hogehoge")
for (Address,Tag_Name) in zip(VPC_CidrBlockList,VPC_TagNameList):
t.add_resource(VPC(
Tag_Name,
EnableDnsSupport="true",
CidrBlock=Address,
EnableDnsHostnames="true",
Tags=Tags(
Name=Tag_Name
)
))
json_template = t.to_json()
print(json_template)
You can now create multiple VPCs according to the parameters listed in the parameter sheet.
The basic part of using Lambda is the same, but the process of getting the parameter sheet from the S3 bucket and the process of uploading the created template to the S3 bucket are added.
python
# coding: utf-8
import json
import urllib
import boto3
from troposphere import Tags,Template
from troposphere.ec2 import VPC
from datetime import datetime
basename = datetime.now().strftime("%Y%m%d-%H%M%S")
print('Loading function')
s3 = boto3.resource('s3')
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
# Get the object from the event and show its content type
bucket = event['Records'][0]['s3']['bucket']['name']
key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key']).decode('utf8')
print ("buket:" + bucket)
print ("key:" + key)
obj = s3.Object(bucket,key)
response= obj.get()
body = response['Body'].read()
body_list= body.splitlines()
# u'Creating a VPC'
VPC_CidrBlockList = body_list[0].split(',')
VPC_TagNameList= body_list[1].split(',')
t = Template()
t.add_version("2010-09-09")
t.add_description("hogehoge")
for (Address,Tag_Name) in zip(VPC_CidrBlockList,VPC_TagNameList):
t.add_resource(VPC(
Tag_Name,
EnableDnsSupport="true",
CidrBlock=Address,
EnableDnsHostnames="true",
Tags=Tags(
Name=Tag_Name
)
))
json_template = t.to_json()
bucket = s3.Bucket('template_test')
obj = bucket.Object('json-template-' + basename + ' .txt')
response = obj.put(
Body=json_template.encode('utf-8'),
ContentEncoding='utf-8',
ContentType='text/plane'
)
print(json_template)
You can also set it more flexibly by changing the items to be set as variables (such as "EnableDnsHostnames" in VPC). Also, if you describe other resources in this way, you can automatically create templates such as subnets and security groups as well as VPCs.
Recommended Posts