Container management with ECS + CodePipline

I think more and more people are using containers when creating applications. This time, I would like to code the mechanism that the Dockerfile is uploaded to CodeCommit and then built to automatically update the ECS task.

Use CloudFormation to manage your infrastructure in code. Refer to the code published by awslabs and create it with Cloudformation. I would like to make it with TerraForm someday.

awslabs https://github.com/awslabs/ecs-refarch-continuous-deployment

The architecture to be created this time is as follows.

EC2 or Fargate can be selected when creating a stack. In addition, AWS official docker image (php-sample) is used when creating a stack, so if you want to use another image, upload the Dockerfile to CodeCommit.

(If you want to use Github for Pipline, please refer to the awslabs code.)

The directory structure is as follows. Please upload the 5 yamls in the templates folder to S3 for use.

├── ecs-refarch-continuous-deployment.yaml
├── tempaltes
    ├── vpc.yaml
    ├── load-balancer.yaml
    ├── ecs-cluster.yaml
    ├── service.yaml
    └── deployment-pipline.yaml

First of all, from VPC.

---
AWSTemplateFormatVersion: 2010-09-09


Parameters:
  Name:
    Type: String

  VpcCIDR:
    Type: String

  Subnet1CIDR:
    Type: String

  Subnet2CIDR:
    Type: String


Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      Tags:
        - Key: Name
          Value: !Ref Name

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref Name

  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  Subnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [ 0, !GetAZs ]
      MapPublicIpOnLaunch: true
      CidrBlock: !Ref Subnet1CIDR
      Tags:
        - Key: Name
          Value: !Sub ${Name} (Public)

  Subnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [ 1, !GetAZs ]
      MapPublicIpOnLaunch: true
      CidrBlock: !Ref Subnet2CIDR
      Tags:
        - Key: Name
          Value: !Sub ${Name} (Public)

  RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Ref Name

  DefaultRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  Subnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref RouteTable
      SubnetId: !Ref Subnet1

  Subnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref RouteTable
      SubnetId: !Ref Subnet2


Outputs:
  Subnets:
    Value: !Join [ ",", [ !Ref Subnet1, !Ref Subnet2 ] ]
  VpcId:
    Value: !Ref VPC

Next is Load Balancer (ALB). There is also an NLB Template, so if you want to see it, please comment!

Parameters:
  LaunchType:
    Type: String
    Default: Fargate
    AllowedValues:
      - Fargate
      - EC2

  Subnets:
    Type: List<AWS::EC2::Subnet::Id>

  VpcId:
    Type: String


Conditions:
  EC2: !Equals [ !Ref LaunchType, "EC2" ]


Resources:
  SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: !Sub ${AWS::StackName}-alb
      SecurityGroupIngress:
        - CidrIp: "0.0.0.0/0"
          IpProtocol: tcp
          FromPort: 80
          ToPort: 80
      VpcId: !Ref VpcId

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Subnets: !Ref Subnets
      SecurityGroups:
        - !Ref SecurityGroup

  LoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref LoadBalancer
      Port: 80
      Protocol: HTTP
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref TargetGroup

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    DependsOn: LoadBalancer
    Properties:
      VpcId: !Ref VpcId
      Port: 80
      Protocol: HTTP
      Matcher:
        HttpCode: 200-299
      HealthCheckIntervalSeconds: 10
      HealthCheckPath: /
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 2
      TargetType: !If [ EC2, "instance", "ip" ]
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: 30

  ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      ListenerArn: !Ref LoadBalancerListener
      Priority: 1
      Conditions:
        - Field: path-pattern
          Values:
            - /
      Actions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward


Outputs:
  TargetGroup:
    Value: !Ref TargetGroup

  ServiceUrl:
    Description: URL of the load balancer for the sample service.
    Value: !Sub http://${LoadBalancer.DNSName}

  SecurityGroup:
    Value: !Ref SecurityGroup

Next is Cluseter. This time the AMI used mapping. You could use SSM to get the latest AMI, but I didn't use it because I wanted to automatically select by region.

Parameters:
  LaunchType:
    Type: String
    Default: Fargate
    AllowedValues:
      - Fargate
      - EC2

  InstanceType:
    Type: String
    Default: t2.micro

  ClusterSize:
    Type: Number
    Default: 2

  Subnets:
    Type: List<AWS::EC2::Subnet::Id>

  SecurityGroup:
    Type: AWS::EC2::SecurityGroup::Id

  VpcId:
    Type: AWS::EC2::VPC::Id

  KeyName:
    Description: The EC2 Key Pair to allow SSH access to the instance
    Type: "AWS::EC2::KeyPair::KeyName"


Conditions:
  EC2: !Equals [ !Ref LaunchType, "EC2" ]

Mappings:
  AWSRegionToAMI:
    ap-south-1:
      AMI: ami-00491f6f
    eu-west-3:
      AMI: ami-9aef59e7
    eu-west-2:
      AMI: ami-67cbd003
    eu-west-1:
      AMI: ami-1d46df64
    ap-northeast-2:
      AMI: ami-c212b2ac
    ap-northeast-1:
      AMI: ami-872c4ae1
    sa-east-1:
      AMI: ami-af521fc3
    ca-central-1:
      AMI: ami-435bde27
    ap-southeast-1:
      AMI: ami-910d72ed
    ap-southeast-2:
      AMI: ami-58bb443a
    eu-central-1:
      AMI: ami-509a053f
    us-east-1:
      AMI: ami-28456852
    us-east-2:
      AMI: ami-ce1c36ab
    us-west-1:
      AMI: ami-74262414
    us-west-2:
      AMI: ami-decc7fa6

Resources:
  EC2Role:
    Type: AWS::IAM::Role
    Condition: EC2
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role

  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Condition: EC2
    Properties:
      Path: /
      Roles:
        - !Ref EC2Role

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Ref AWS::StackName

  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Condition: EC2
    Properties:
      VPCZoneIdentifier: !Ref Subnets
      LaunchConfigurationName: !Ref LaunchConfiguration
      MinSize: !Ref ClusterSize
      MaxSize: !Ref ClusterSize
      DesiredCapacity: !Ref ClusterSize
      Tags: 
        - Key: Name
          Value: !Sub ${AWS::StackName} - ECS Host
          PropagateAtLaunch: true
    CreationPolicy:
      ResourceSignal:
        Timeout: PT15M
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: 1
        MaxBatchSize: 1
        PauseTime: PT15M
        WaitOnResourceSignals: true

  LaunchConfiguration:
    Type: AWS::AutoScaling::LaunchConfiguration
    Condition: EC2
    Metadata:
      AWS::CloudFormation::Init:
        config:
          commands:
            01_add_instance_to_cluster:
                command: !Sub echo ECS_CLUSTER=${Cluster} > /etc/ecs/ecs.config
          files:
            "/etc/cfn/cfn-hup.conf":
              mode: 000400
              owner: root
              group: root
              content: !Sub |
                [main]
                stack=${AWS::StackId}
                region=${AWS::Region}
            "/etc/cfn/hooks.d/cfn-auto-reloader.conf":
              content: !Sub |
                [cfn-auto-reloader-hook]
                triggers=post.update
                path=Resources.ContainerInstances.Metadata.AWS::CloudFormation::Init
                action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchConfiguration
          services:
            sysvinit:
              cfn-hup:
                enabled: true
                ensureRunning: true
                files:
                  - /etc/cfn/cfn-hup.conf
                  - /etc/cfn/hooks.d/cfn-auto-reloader.conf
    Properties:
      ImageId: !FindInMap [ AWSRegionToAMI, !Ref "AWS::Region", AMI ]
      InstanceType: !Ref InstanceType
      IamInstanceProfile: !Ref InstanceProfile
      SecurityGroups:
        - !Ref SecurityGroup
      KeyName: !Ref KeyName
      UserData:
        "Fn::Base64": !Sub |
          #!/bin/bash
          yum install -y aws-cfn-bootstrap
          /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchConfiguration
          /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource AutoScalingGroup

Outputs:
  ClusterName:
      Value: !Ref Cluster

Next is Service.

Parameters:
  Cluster:
    Type: String

  DesiredCount:
    Type: Number
    Default: 1

  LaunchType:
    Type: String
    Default: Fargate
    AllowedValues:
      - Fargate
      - EC2

  TargetGroup:
    Type: String

  SecurityGroup:
    Type: AWS::EC2::SecurityGroup::Id

  Subnets:
    Type: List<AWS::EC2::Subnet::Id>


Conditions:
  Fargate: !Equals [ !Ref LaunchType, "Fargate" ]

  EC2: !Equals [ !Ref LaunchType, "EC2" ]


Resources:
  TaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub /ecs/${AWS::StackName}
      RetentionInDays: 14

  FargateService:
    Type: AWS::ECS::Service
    Condition: Fargate
    Properties:
      Cluster: !Ref Cluster
      DesiredCount: !Ref DesiredCount
      TaskDefinition: !Ref TaskDefinition
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: ENABLED
          SecurityGroups:
            - !Ref SecurityGroup
          Subnets: !Ref Subnets
      LoadBalancers:
        - ContainerName: simple-app
          ContainerPort: 80
          TargetGroupArn: !Ref TargetGroup

  EC2Service:
    Type: AWS::ECS::Service
    Condition: EC2
    Properties:
      Cluster: !Ref Cluster
      DesiredCount: !Ref DesiredCount
      TaskDefinition: !Ref TaskDefinition
      LaunchType: EC2
      LoadBalancers:
        - ContainerName: simple-app
          ContainerPort: 80
          TargetGroupArn: !Ref TargetGroup

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Sub ${AWS::StackName}-simple-app
      RequiresCompatibilities:
        - !If [ Fargate, "FARGATE", "EC2" ]
      Memory: 512
      Cpu: 256
      NetworkMode: !If [ Fargate, "awsvpc", "bridge" ]
      ExecutionRoleArn: !Ref TaskExecutionRole
      ContainerDefinitions:
        - Name: simple-app
          Image: amazon/amazon-ecs-sample
          Essential: true
          Memory: 256
          MountPoints:
            - SourceVolume: my-vol
              ContainerPath: /var/www/my-vol
          PortMappings:
            - HostPort: 80
              ContainerPort: 80
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-region: !Ref AWS::Region
              awslogs-group: !Ref LogGroup
              awslogs-stream-prefix: !Ref AWS::StackName
        - Name: busybox
          Image: busybox
          EntryPoint:
            - sh
            - -c
          Essential: true
          Memory: 256
          VolumesFrom:
            - SourceContainer: simple-app
          Command:
            - /bin/sh -c "while true; do /bin/date > /var/www/my-vol/date; sleep 1; done"
      Volumes:
        - Name: my-vol


Outputs:
  Service:
    Value: !If [ Fargate, !Ref FargateService, !Ref EC2Service ]

Next is the construction of the pipeline.

Parameters:
  CodeCommitRepositoryName:
    Type: String

  Cluster:
    Type: String

  Service:
    Type: String


Resources:
  Repository:
    Type: AWS::ECR::Repository
    DeletionPolicy: Retain

  CodeBuildServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: codebuild.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Resource: "*"
                Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                  - ecr:GetAuthorizationToken
                  - secretsmanager:GetSecretValue
              - Resource: !Sub arn:aws:s3:::${ArtifactBucket}/*
                Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:GetObjectVersion
              - Resource: !Sub arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${Repository}
                Effect: Allow
                Action:
                  - ecr:GetDownloadUrlForLayer
                  - ecr:BatchGetImage
                  - ecr:BatchCheckLayerAvailability
                  - ecr:PutImage
                  - ecr:InitiateLayerUpload
                  - ecr:UploadLayerPart
                  - ecr:CompleteLayerUpload

  CodePipelineServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: codepipeline.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Resource:
                  - !Sub arn:aws:s3:::${ArtifactBucket}/*
                Effect: Allow
                Action:
                  - s3:PutObject
                  - s3:GetObject
                  - s3:GetObjectVersion
                  - s3:GetBucketVersioning
              - Resource: "*"
                Effect: Allow
                Action:
                  - ecs:DescribeServices
                  - ecs:DescribeTaskDefinition
                  - ecs:DescribeTasks
                  - ecs:ListTasks
                  - ecs:RegisterTaskDefinition
                  - ecs:UpdateService
                  - codebuild:StartBuild
                  - codebuild:BatchGetBuilds
                  - iam:PassRole

  ArtifactBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain

  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: CODEPIPELINE
      Source:
        Type: CODEPIPELINE
        BuildSpec: |
          version: 0.2
          phases:
            pre_build:
              commands:
                - $(aws ecr get-login --no-include-email)
                - TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)"
                - IMAGE_URI="${REPOSITORY_URI}:${TAG}"
            build:
              commands:
                - docker build --tag "$IMAGE_URI" .
            post_build:
              commands:
                - docker push "$IMAGE_URI"
                - printf '[{"name":"simple-app","imageUri":"%s"}]' "$IMAGE_URI" > images.json
          artifacts:
            files: images.json
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/docker:17.09.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: AWS_DEFAULT_REGION
            Value: !Ref AWS::Region
          - Name: REPOSITORY_URI
            Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${Repository}
      Name: !Ref AWS::StackName
      ServiceRole: !Ref CodeBuildServiceRole

  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineServiceRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref ArtifactBucket
      Stages:
        - Name: Source
          Actions:
            - Name: App
              ActionTypeId:
                Category: Source
                Owner: AWS
                Version: 1
                Provider: CodeCommit
              Configuration:
                RepositoryName: !Ref CodeCommitRepositoryName
                BranchName: master
              RunOrder: 1
              OutputArtifacts:
                - Name: App
        - Name: Build
          Actions:
            - Name: Build
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: 1
                Provider: CodeBuild
              Configuration:
                ProjectName: !Ref CodeBuildProject
              InputArtifacts:
                - Name: App
              OutputArtifacts:
                - Name: BuildOutput
              RunOrder: 1
        - Name: Deploy
          Actions:
            - Name: Deploy
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: 1
                Provider: ECS
              Configuration:
                ClusterName: !Ref Cluster
                ServiceName: !Ref Service
                FileName: images.json
              InputArtifacts:
                - Name: BuildOutput
              RunOrder: 1


Outputs:
  PipelineUrl:
    Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}

Please upload the above 5 files to S3. Finally, a summary template.

Parameters:
  LaunchType:
    Type: String
    Default: Fargate
    AllowedValues:
      - Fargate
      - EC2
    Description: >
      The launch type for your service. Selecting EC2 will create an Auto
      Scaling group of t2.micro instances for your cluster. See
      https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_types.html
      to learn more about launch types.

  TemplateBucket:
    Type: String
    Description: >
      The S3 bucket from which to fetch the templates used by this stack.
    
  CodeCommitRepositoryName:
    Type: String

  ClusterSize:
    Type: Number
    Default: 2

  DesiredCount:
    Type: Number
    Default: 1

  KeyName:
    Description: The EC2 Key Pair to allow SSH access to the instance
    Type: "AWS::EC2::KeyPair::KeyName"

Metadata:
  AWS::CloudFormation::Interface:
    ParameterLabels:
      CodeCommitRepositoryName:
        default: "CodeCommitRepositoryName"
      LaunchType:
        default: "Launch Type"
    ParameterGroups:
      - Label:
          default: Cluster Configuration
        Parameters:
          - LaunchType
      - Label:
          default: CodeCommit Configuration
        Parameters:
          - CodeCommitRepositoryName
      - Label:
          default: Stack Configuration
        Parameters:
          - TemplateBucket
      - Label:
          default: TaskDesiredCount
        Parameters:
          - DesiredCount

Resources:
  Cluster:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://s3.amazonaws.com/${TemplateBucket}/ecs-cluster.yaml"
      Parameters:
        LaunchType: !Ref LaunchType
        SecurityGroup: !GetAtt LoadBalancer.Outputs.SecurityGroup
        Subnets: !GetAtt VPC.Outputs.Subnets
        VpcId: !GetAtt VPC.Outputs.VpcId
        ClusterSize: !Ref ClusterSize
        KeyName: !Ref KeyName

  DeploymentPipeline:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://s3.amazonaws.com/${TemplateBucket}/deployment-pipeline.yaml"
      Parameters:
        Cluster: !GetAtt Cluster.Outputs.ClusterName
        Service: !GetAtt Service.Outputs.Service
        CodeCommitRepositoryName: !Ref CodeCommitRepositoryName

  LoadBalancer:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://s3.amazonaws.com/${TemplateBucket}/load-balancer.yaml"
      Parameters:
        LaunchType: !Ref LaunchType
        Subnets: !GetAtt VPC.Outputs.Subnets
        VpcId: !GetAtt VPC.Outputs.VpcId

  VPC:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://s3.amazonaws.com/${TemplateBucket}/vpc.yaml"
      Parameters:
        Name: !Ref AWS::StackName
        VpcCIDR: 10.215.0.0/16
        Subnet1CIDR: 10.215.10.0/24
        Subnet2CIDR: 10.215.20.0/24

  Service:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub "https://s3.amazonaws.com/${TemplateBucket}/service.yaml"
      Parameters:
        Cluster: !GetAtt Cluster.Outputs.ClusterName
        LaunchType: !Ref LaunchType
        TargetGroup: !GetAtt LoadBalancer.Outputs.TargetGroup
        SecurityGroup: !GetAtt LoadBalancer.Outputs.SecurityGroup
        Subnets: !GetAtt VPC.Outputs.Subnets
        DesiredCount: !Ref DesiredCount

Outputs:
  ServiceUrl:
    Description: The sample service that is being continuously deployed.
    Value: !GetAtt LoadBalancer.Outputs.ServiceUrl

  PipelineUrl:
    Description: The continuous deployment pipeline in the AWS Management Console.
    Value: !GetAtt DeploymentPipeline.Outputs.PipelineUrl

That is all. Thank you for watching until the end.

Reference article https://qiita.com/chisso/items/3c4dd1af0382d4978288 https://github.com/awslabs/ecs-refarch-continuous-deployment