[PYTHON] AWS Official "Building Modern Web Applications" Hands-on Hamari

AWS publishes a lot of practical hands-on content that utilizes AWS services. It is also translated into Japanese.

Amazon Web Services (AWS) Practical Tutorial https://aws.amazon.com/jp/getting-started/hands-on/

When I was looking for a container for internal use and hands-on teaching materials for API Gateway beginners, I found the right tutorial.

Build a modern web application https://aws.amazon.com/jp/getting-started/hands-on/build-modern-app-fargate-lambda-dynamodb-python/

This tutorial,

-** Publication of container application utilizing ECS / Fargate ** -** Construction of CI / CD pipeline utilizing Code service ** -** Publication of API utilizing Amazon API Gateway ** -** Using DynamoDB as a data store ** -** Addition of authentication processing using Cognito ** -** Clickstream analysis using Kinesis Firehose **

You can experience the construction of a modern application execution platform step by step, such as Very good content.

Source: https://aws.amazon.com/jp/getting-started/hands-on/build-modern-app-fargate-lambda-dynamodb-python/

Good content ... very good content ...

Even if I tried it according to the procedure of the tutorial, there were some points that did not work well, so I would like to make a note. Resources for .NET, Go, and Java are also available, but this article is a reminder of my trial with Python.

Module 2B: Deploying Services Using AWS Fargate

Step 1: Create a Flask service

A: Build the Docker image

I do docker build on Cloud9 IDE ...

docker build . -t REPLACE_ME_AWS_ACCOUNT_ID.dkr.ecr.REPLACE_ME_REGION.amazonaws.com/mythicalmysfits/service:latest

python-Installing pip or pip will moss.

Building dependency tree...
Reading state information...
E: Unable to locate package python-pip
The command '/bin/sh -c apt-get install -y python-pip python-dev build-essential' returned a non-zero code: 100

Step 5/13 : RUN pip install --upgrade pip
 ---> Running in 944121289de0
/bin/sh: 1: pip: not found
The command '/bin/sh -c pip install --upgrade pip' returned a non-zero code: 127

As of August 2020, ubuntu: latest Python is v3 series, so Let's make Dockerfile compatible with python3.

python-pip is now ** python3-pip **, pip to ** pip3 **, Change python in ENTRYPOINT to ** python3 **.

FROM ubuntu:latest
RUN echo Updating existing packages, installing and upgrading python and pip.
RUN apt-get update -y
RUN apt-get install -y python3-pip python-dev build-essential
RUN pip3 install --upgrade pip
RUN echo Copying the Mythical Mysfits Flask service into a service directory.
COPY ./service /MythicalMysfitsService
WORKDIR /MythicalMysfitsService
RUN echo Installing Python packages listed in requirements.txt
RUN pip3 install -r ./requirements.txt
RUN echo Starting python and starting the Flask service...
ENTRYPOINT ["python3"]
CMD ["mythicalMysfitsService.py"]

Let's modify the Dockerfile provided in module-3 and later in the same way.

C: Push Docker image to Amazon ECR

You need to log in before docker push to ECR ...

$(aws ecr get-login --no-include-email)

get-login is deprecated, so use ** get-login-password **.

• Replace AWS_ACCOUNT_ID with your AWS account number.

aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin AWS_ACCOUNT_ID.dkr.ecr.ap-northeast-1.amazonaws.com

Module 2C: Automation of deployment using AWS code services

Step 1: Create a CI / CD pipeline

C: Create a CodeBuild project

Running the provided AWS CLI will create a CodeBuild project, If you move the CI / CD pipeline in step 2, the build will moss.

The cause is that the above deprecated get-login is used in buildspec.yml. Instead, use ** aws ecr get-login-password **.

Open file: //~/environment/aws-modern-application-workshop/module-2/aws-cli/ecr-policy.json and open pre_build phase

- $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION)

To

- aws ecr get-login-password --region $AWS_DEFAULT_REGION) | docker login --username AWS --password-stdin  $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com

Let's fix it to.

• For \ $ AWS_ACCOUNT_ID and \ $ AWS_DEFAULT_REGION, set the values that match your environment in the environment variable definition.

Module 3: Save Mysfit Information

Step 2: Commit the first actual code change

A: Copy the updated Flask service code

Resources to push to CodeCommit are prepared in advance, Copy it to the directory for the repository.

cp -r ~/environment/aws-modern-application-workshop/module-2/app/* ~/environment/MythicalMysfitsService-Repository/

Pushing this resource to CodeCommit will deploy the app to ECS, The task will drop.

2020-08-30 16:22:28Traceback (most recent call last):
2020-08-30 16:22:28File "mythicalMysfitsService.py", line 3, in <module>
2020-08-30 16:22:28import mysfitsTableClient
2020-08-30 16:22:28File "/MythicalMysfitsService/mysfitsTableClient.py", line 104
2020-08-30 16:22:28print 'filter is '+args.filter
2020-08-30 16:22:28^
2020-08-30 16:22:28SyntaxError: Missing parentheses in call to 'print'. Did you mean print('filter is '+args.filter)?

Looking at the ECS task log, it's down at the print function in mysfitsTableClient.py.

This tutorial seems to have been developed using Python 2.7. Since the description of the print function is different in Python3 specified in the Dockerfile, Modify the app.

Before correction

    if args.filter and args.value:
        print 'filter is '+args.filter
        print 'value is '+args.value

        print "Getting filtered values"
        items = queryMysfitItems(args.filter, args.value)
    else:
        print "Getting all values"
        items = getAllMysfits()

    print items

Revised

• Enclose the argument of the print function in ()

    if args.filter and args.value:
        print('filter is '+args.filter)
        print('value is '+args.value)

        print("Getting filtered values")
        items = queryMysfitItems(args.filter, args.value)
    else:
        print("Getting all values")
        items = getAllMysfits()

    print(items)

Module 4: User Registration Settings

Step 2: Add a new REST API and Amazon API Gateway

B: Create a REST API using Swagger

I'm trying to create a REST API by importing an API definition from a Swagger definition ...

aws apigateway import-rest-api --parameters endpointConfigurationTypes=REGIONAL --body file://~/environment/aws-modern-application-workshop/module-4/aws-cli/api-swagger.json --fail-on-warnings

I don't get a JSON response like the one in the tutorial procedure. The return value of the shell is 255 and the API Gateway is not created either.

First, I doubted the Swagger definition. You can use https://editor.swagger.io/ or VS Code's OpenAPI Editor plugin, If you check the definition contents of api-swagger.json, POST / mysfits / {mysfitId} / like definition, x-amazon-apigateway-integration part You can see that the responses are duplicated.

After all, I will erase the duplicate responses, but The result of aws apigateway import-rest-api does not change ... why?

In this case, run it with the --debug option.

aws apigateway import-rest-api --parameters endpointConfigurationTypes=REGIONAL --body file://~/environment/aws-modern-application-workshop/module-4/aws-cli/api-swagger.json --fail-on-warnings --debug

Then ~? ?? ??

2020-08-31 06:39:10,907 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.0.11 Python/3.7.3 Linux/4.14.181-142.260.amzn2.x86_64 botocore/2.0.0dev15
(Omission)
2020-08-31 06:39:10,994 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/customizations/binaryformat.py", line 59, in _visit_scalar
  File "base64.py", line 87, in b64decode
binascii.Error: Invalid base64-encoded string: number of data characters (6953) cannot be 1 more than a multiple of 4
(Omission)
  File "awscli/customizations/binaryformat.py", line 34, in base64_decode_input_blobs
  File "awscli/shorthand.py", line 389, in visit
  File "awscli/shorthand.py", line 394, in _visit
  File "awscli/shorthand.py", line 401, in _visit_structure
  File "awscli/shorthand.py", line 394, in _visit
  File "awscli/customizations/binaryformat.py", line 61, in _visit_scalar
awscli.customizations.binaryformat.InvalidBase64Error: Invalid base64: "{
    "swagger": 2.0,
    "info": {
        "title": "MysfitsApi"

Invalid base64？

Let's see the changes from AWS CLI v1 to v2.

Important Changes – Migrating from AWS CLI Version 1 to Version 2 --AWS Command Line Interface https://docs.aws.amazon.com/ja_jp/cli/latest/userguide/cliv2-migration.html#cliv2-migration-binaryparam

Oh! !! !!

AWS CLI version 2 now passes all binary input and output parameters as base64 encoded strings by default. (Omitted) You can also override the active profile settings and undo the settings for individual commands by including the parameter --cli-binary-format raw-in-base64-out on the command line.

I see. The Swagger definition file is not base64 encoded, so ** --cli-binary-format raw-in-base64-out ** Let's add a parameter.

aws apigateway import-rest-api --cli-binary-format raw-in-base64-out --parameters endpointConfigurationTypes=REGIONAL --body file://~/environment/aws-modern-application-workshop/module-4/aws-cli/api-swagger.json --fail-on-warnings --debug

I passed.

{
    "id": "6xxxxxxxx3",
    "name": "MysfitsApi",
    "createdDate": "2020-08-31T07:29:58+00:00",
    "version": "2017-04-20T04:08:08Z",
    "apiKeySource": "HEADER",
    "endpointConfiguration": {
        "types": [
            "REGIONAL"
        ]
    }
}

I was able to complete the rest of the steps without any particular problems.

I will post an issue on Github for the parts that need to be updated such as AWS CLI and Dockerfile, It will take some time to be reflected on the tutorial website (especially the Japanese version), so Please refer to it as an auxiliary document.

You can experience cloud-native application building step by step, so Those who want to learn ECS, API Gateway, Cognito, Kinesis Firehose, etc. while moving their hands, Please try it! !! !!