[PYTHON] code-server Online environment (2) Create a virtual network with Boto3

This is the 13th day article of Advent Calender on the 2019 code-server.

Continuing from the last time, I would like to launch EC2 Instance.

table of contents Local environment 1st day Online environment version 1st day Improve work environment

Online environment, day 2 Create a virtual network

Online environment 3rd day Launch an EC2 instance with Boto3

Online environment 4th day Try running Code-Server in the cloud

Online environment 5th day Launch code-server on Docker

Online environment, day 6 Let's automate

Online environment 7th day Deploy compute on git on EC2

... Online version .. Built with Coompose file

Online .. Try K8S

...

Demon remodeling

What is EC2

https://aws.amazon.com/ec2/

EC2 is a virtual server provided by AWS. A server that charges in seconds. You can use Linux freely.

Let's build a network !!

Before launching EC2 Instance, let's build a network. You can use the default one, but please try it.

Create a virtual network

https://aws.amazon.com/vpc/

First, create a virtual network. ..

import boto3
from boto3_type_annotations import ec2

instance_name= "advent-code-server"

ec2client:ec2.Client = boto3.client("ec2")
res = ec2client.create_vpc(CidrBlock='10.1.0.0/16')
print("{}".format(res))

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.create_vpc

With just this, you can create it. CidrBlock means the IP to allocate. In this case IPs from 10.1.0.0 to 10.1.255.255 are available.

Let's hit TAG for easy management

import boto3
from boto3_type_annotations import ec2
from typing import Dict, List 

instance_name= "advent-code-server"

def attach_tag(id:str):
    res = ec2client.create_tags(Resources=[id], Tags=[{"Key": "Name", "Value": instance_name}])
    print("{}".format(res))

def create_vpc():
    print(">>> CREATE VPC")
    res = ec2client.create_vpc(CidrBlock='10.1.0.0/16')
    print("{}".format(res))
    vpc_id = res['Vpc']['VpcId']
    attach_tag(vpc_id)
    return vpc_id

Yes, it's done. You can create a virtual network by calling create_vpc ().

Delete virtual network

You need to be able to delete what you have created at any time. Let's write a script to delete

def rm_vpc():
    print(">>> Delete vpcs")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.delete_vpc(VpcId=vpc['VpcId'])
        print("{}".format(res))

Since Tag is attached, it is easy to delete. Find the VPC associated with the Tag and delete the found VPC.

Let's try so far.

import boto3
from boto3_type_annotations import ec2
from botocore.exceptions import ClientError
from typing import Dict, List 

instance_name= "advent-code-server"
ec2client:ec2.Client = boto3.client("ec2")

def attach_tag(id:str):
    res = ec2client.create_tags(Resources=[id], Tags=[{"Key": "Name", "Value": instance_name}])
    print("{}".format(res))

def create_vpc():
    print(">>> CREATE VPC")
    res = ec2client.create_vpc(CidrBlock='10.1.0.0/16')
    print("{}".format(res))
    vpc_id = res['Vpc']['VpcId']
    attach_tag(vpc_id)
    return vpc_id

def delete_vpc():
    print(">>> Delete vpcs")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.delete_vpc(VpcId=vpc['VpcId'])
        print("{}".format(res))
    
if __name__ == "__main__":
    create_vpc()
    rm_vpc()

Now you have the code to create and delete!

　　　　 ∧＿∧
/ \ (・ ∀ ・) / ヽ
(● and Tsu ●)..break..
　＼/⊂, no \ no
Shi ’

Added Internet Gateway

https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html

Your VPC will not be able to connect to the Internet at this point. Let's set up a gateway to connect.

def create_gateway(vpc_id:str):
    print(">>> CREATE GATEWAY")
    res = ec2client.create_internet_gateway()
    print("{}".format(res))
    gateway_id = res['InternetGateway']['InternetGatewayId']
    attach_tag(gateway_id)

    print(">>> ATTACH GATEWAY")
    res = ec2client.attach_internet_gateway(InternetGatewayId=gateway_id,VpcId=vpc_id)
    print("{}".format(res))

I've created a Gateway and associated it with a VPC.

Remove Internet Gateway

Now let's write the code to remove.

def delete_gateway():
    print(">> Detach Gateway")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
        print("{}".format(res))
        for gateway in res['InternetGateways']:
            res = ec2client.detach_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'],VpcId=vpc['VpcId'])
            print("{}".format(res))

    print(">> Delete Gateway")
    res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for gateway in res['InternetGateways']:
        res = ec2client.delete_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'])
        print("{}".format(res))

You need to disassociate your VPC to remove it. After that, in the same way, search for the data associated with the Tag and delete it.

Let's try so far.

import boto3
from boto3_type_annotations import ec2
from botocore.exceptions import ClientError
from typing import Dict, List 

instance_name= "advent-code-server"
ec2client:ec2.Client = boto3.client("ec2")



def attach_tag(id:str):
    res = ec2client.create_tags(Resources=[id], Tags=[{"Key": "Name", "Value": instance_name}])
    print("{}".format(res))

def create_vpc():
    print(">>> CREATE VPC")
    res = ec2client.create_vpc(CidrBlock='10.1.0.0/16')
    print("{}".format(res))
    vpc_id = res['Vpc']['VpcId']
    attach_tag(vpc_id)
    return vpc_id

def delete_vpc():
    print(">>> Delete vpcs")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.delete_vpc(VpcId=vpc['VpcId'])
        print("{}".format(res))
    
def create_gateway(vpc_id:str):
    print(">>> CREATE GATEWAY")
    res = ec2client.create_internet_gateway()
    print("{}".format(res))
    gateway_id = res['InternetGateway']['InternetGatewayId']
    attach_tag(gateway_id)

    print(">>> ATTACH GATEWAY")
    res = ec2client.attach_internet_gateway(InternetGatewayId=gateway_id,VpcId=vpc_id)
    print("{}".format(res))

def delete_gateway():
    print(">> Detach Gateway")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
        print("{}".format(res))
        for gateway in res['InternetGateways']:
            res = ec2client.detach_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'],VpcId=vpc['VpcId'])
            print("{}".format(res))

    print(">> Delete Gateway")
    res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for gateway in res['InternetGateways']:
        res = ec2client.delete_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'])
        print("{}".format(res))

if __name__ == "__main__":
    vpc_id:str = create_vpc()
    gateway_id:str = create_gateway(vpc_id)
    delete_gateway()
    delete_vpc()

Now you have the code to create and delete!

　　　　 ∧＿∧
/ \ (・ ∀ ・) / ヽ
(● and Tsu ●)..break..
　＼/⊂, no \ no
Shi ’

Set subnet

https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html

Let's set the subnet. Subnet is the network set in VPC, It can be further divided, grouped and managed.

Create

def create_subnet(vpc_id:str):
    print(">>> CREATE SUBNET")
    res = ec2client.create_subnet(CidrBlock='10.1.0.0/24',VpcId=vpc_id)
    print("{}".format(res))
    subnet_id = res['Subnet']['SubnetId']
    attach_tag(subnet_id)
    return subnet_id

Delete

def delete_subnet():
    print(">> Delete subnet")
    res = ec2client.describe_subnets(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for subnet in res["Subnets"]:
        res = ec2client.delete_subnet(SubnetId=subnet['SubnetId'])
        print("{}".format(res))

CidrBlock means the IP to allocate. In this case IPs from 10.1.0.0 to 10.1.0.255 are available.

Set security group

Let's set security such as which port to release and which port to close.

Create

def create_security_group():
    print(">>> CREATE SECURITY GROUP")
    res = ec2client.create_security_group(Description="AdventCodeServer",GroupName=instance_name)
    print("{}".format(res))
    group_id = res['GroupId']
    attach_tag(group_id)
    return group_id

Delete

def delete_security_group():
    res = ec2client.describe_security_groups(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for sg in res['SecurityGroups']:
        res = ec2client.delete_security_group(GroupId=sg["GroupId"])
        print("{}".format(res))

Added Port settings

def create_security_group_ingress():
        print(">>>> CREATE SECURITY GROUP INGRESS")
        res = ec2client.authorize_security_group_ingress(
                GroupName=instance_name, IpPermissions=[
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 8443,
                        'ToPort': 8443,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8443'}
                        ]
                    },
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 8080,
                        'ToPort': 8080,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8080'}
                        ]
                    },
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 22,
                        'ToPort': 22,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8080'}
                        ]
                    },
                ])
        print("{}".format(res))

Let's try so far.

import boto3
from boto3_type_annotations import ec2
from botocore.exceptions import ClientError
from typing import Dict, List 

instance_name= "advent-code-server"
ec2client:ec2.Client = boto3.client("ec2")



def attach_tag(id:str):
    res = ec2client.create_tags(Resources=[id], Tags=[{"Key": "Name", "Value": instance_name}])
    print("{}".format(res))

def create_vpc():
    print(">>> CREATE VPC")
    res = ec2client.create_vpc(CidrBlock='10.1.0.0/16')
    print("{}".format(res))
    vpc_id = res['Vpc']['VpcId']
    attach_tag(vpc_id)
    return vpc_id

def delete_vpc():
    print(">>> Delete vpcs")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.delete_vpc(VpcId=vpc['VpcId'])
        print("{}".format(res))
    
def create_gateway(vpc_id:str):
    print(">>> CREATE GATEWAY")
    res = ec2client.create_internet_gateway()
    print("{}".format(res))
    gateway_id = res['InternetGateway']['InternetGatewayId']
    attach_tag(gateway_id)

    print(">>> ATTACH GATEWAY")
    res = ec2client.attach_internet_gateway(InternetGatewayId=gateway_id,VpcId=vpc_id)
    print("{}".format(res))

def delete_gateway():
    print(">> Detach Gateway")
    res = ec2client.describe_vpcs(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for vpc in res["Vpcs"]:
        res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
        print("{}".format(res))
        for gateway in res['InternetGateways']:
            res = ec2client.detach_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'],VpcId=vpc['VpcId'])
            print("{}".format(res))

    print(">> Delete Gateway")
    res = ec2client.describe_internet_gateways(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for gateway in res['InternetGateways']:
        res = ec2client.delete_internet_gateway(InternetGatewayId=gateway['InternetGatewayId'])
        print("{}".format(res))

def create_subnet(vpc_id:str):
    print(">>> CREATE SUBNET")
    res = ec2client.create_subnet(CidrBlock='10.1.0.0/24',VpcId=vpc_id)
    print("{}".format(res))
    subnet_id = res['Subnet']['SubnetId']
    attach_tag(subnet_id)
    return subnet_id

def delete_subnet():
    print(">> Delete subnet")
    res = ec2client.describe_subnets(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for subnet in res["Subnets"]:
        res = ec2client.delete_subnet(SubnetId=subnet['SubnetId'])
        print("{}".format(res))

def create_security_group():
    print(">>> CREATE SECURITY GROUP")
    res = ec2client.create_security_group(Description="AdventCodeServer",GroupName=instance_name)
    print("{}".format(res))
    group_id = res['GroupId']
    attach_tag(group_id)
    return group_id

def delete_security_group():
    res = ec2client.describe_security_groups(Filters=[{"Name":"tag:Name","Values":[instance_name]}])
    print("{}".format(res))
    for sg in res['SecurityGroups']:
        res = ec2client.delete_security_group(GroupId=sg["GroupId"])
        print("{}".format(res))

def create_security_group_ingress():
        print(">>>> CREATE SECURITY GROUP INGRESS")
        res = ec2client.authorize_security_group_ingress(
                GroupName=instance_name, IpPermissions=[
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 8443,
                        'ToPort': 8443,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8443'}
                        ]
                    },
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 8080,
                        'ToPort': 8080,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8080'}
                        ]
                    },
                    {
                        'IpProtocol': 'tcp',
                        'FromPort': 22,
                        'ToPort': 22,
                        'IpRanges':[
                            {'CidrIp': '0.0.0.0/0', 'Description' : '8080'}
                        ]
                    },
                ])
        print("{}".format(res))

if __name__ == "__main__":
    vpc_id:str = create_vpc()
    gateway_id:str = create_gateway(vpc_id)
    subnet_id = create_subnet(vpc_id)
    group_id = create_security_group()
    create_security_group_ingress()
    #create_instance()
    #delete_instance()
    delete_security_group()
    delete_subnet()
    delete_gateway()
    delete_vpc()

Now you have the code to create and delete! This completes the network settings.

next time

Let's start a virtual server on the created virtual network !!

code

https://github.com/kyorohiro/advent-2019-code-server/tree/master/remote_cs01