Dynamic HTML pages made with AWS Lambda and Python

Thing you want to do

I want to create a page that generates dynamic HTML using AWS Lambda __. With Lambda, you can run your favorite Python scripts without having to set up your own server, so I'll try to use it to create a page that generates dynamic HTML.

There are already quite a few talks about creating pages with Lambda, but the previously announced __API Gateway Lambda proxy integration __ The purpose of this time is to make it feel a little better by combining Lambda's deployment tool apex. Also, template engine is indispensable for spitting HTML from scripts, so I played with it to verify whether they can actually be used in combination.

What to make

As for what kind of page to make, this time I decided to build a __ shrine __. There is a famous template engine of Python called jinja2, and I decided to create a serverless shrine for it. (But it's just a web page, just in case)

I want to make it as a sample of a dynamic page, so I will set up an access counter that shows the number of worshipers. Well, returning the current time is a good example of dynamics, but it's also dull to do what you can do with Javascript on the server, and I think that most of the things that require dynamic processing are for messing around with the database. , I chose the access counter as a practical element. Keep an eye out for the fact that the access counter itself is treated like a fossil.

Diagram

https://cloudcraft.co/view/67eed248-a460-46dc-91dd-fdb5a70f529f?key=D4aRY_6kyWIfbEKtYk78TQ

The configuration of the server I will make this time is like this. Lambda can't be hit directly from the browser, so I've bitten the API Gateway before to access it. In addition, DynamoDB is installed behind it to save the number of accesses. DynamoDB is also a key-value database that can be used without building an instance. It's simple, but it's still a great serverless architecture.

Preparation

Apex

• http://apex.run/

This time I decided to deploy to Lambda using a tool called Apex. If you want to use external libraries with Lambda, you need to zip them up and place them, but this tool called Apex is very convenient because you can easily deploy and execute the installed scripts from the command line.

I won't go into details here,

1. Install Apex
2. Place the AWS key in an environment variable
3. Initialize the project and create a template with apex init
4. Implement the function with reference to the template
5. apex deploy

You can deploy quickly like this.

DynamoDB Prepare a table for access aggregation first. In the case of DynamoDB, it is not necessary to define all the columns to be used because it is schemaless, but it is necessary to decide the primary key for scanning in advance.

This time, I made the following table.

We have a primary partition key named counter_id. To use it, prepare 10 counters with counter_ids from 0 to 9, specify them randomly, and increment them. By preparing multiple counters, writing is not concentrated in one place and constant performance can be maintained.

Since DynamoDB has no transactions, I thought that writing to the same table at the same time would make it inconsistent, but it seems that an atomic counter can be realized by incrementing as an expression using ʻupdate_item`. (I just learned)

When reading, we will get all the counters and return the sum of them. If you don't use "strong consistency", it seems that the written result will not be reflected immediately, but this time we are not asking for strictness, so we use reasonable weak consistency.

The result of writing several times.

Implementation

I will make it immediately.

Using Jinja2 with Lambda

By putting the Jinja2 package and the template I want to use into the Apex function folder, I was able to use the template engine with Lambda without any problems.

Folder structure.txt

├── functions
│   └── jinja
│       ├── jinja2/Such
│       ├── main.py
│       ├── requirements.txt
│       └── templates
│           ├── index.html
│           └── layout.html
├── project.json

The folder structure seen from the Apex project root is above. There is a folder called jinja in the folder called functions, which is the root folder of the Lambda function defined this time. By creating multiple folders in this hierarchy, it is possible to handle different Lambda functions together.

In order to use jinja2, you need to run pip install -t .jinja2 in the jinja folder to install the entire jinja2 package. Also, create a folder for the template files you plan to use.

main.py

# coding: utf-8

from jinja2 import Environment, FileSystemLoader

env = Environment(loader=FileSystemLoader(path.join(path.dirname(__file__), 'templates'), encoding='utf8'))

template = env.get_template('index.html')
html = template.render()

When calling, it looks like this. You can now output the templates / index.html template.

Access DynamoDB

It's easy to use the official library boto3 to access DynamoDB from your Lambda function. You can use it without putting it in the Apex package (in some cases, you put it explicitly because you want to fix the version), and if you specify the appropriate Role for Lambda, you can use it without having to authenticate in your code.

main.py

import boto3

dynamodb = boto3.resource('dynamodb')
count_table  = dynamodb.Table('lambda-jinja')
counts = count_table.scan(Limit=10)

API Gateway Lambda proxy integration

With API Gateway, you can run Lambda from HTTP and receive the results. However, with the conventional API Gateway, it was necessary to map what kind of request came in what kind of path and what kind of response was returned. This is ~~ relatively troublesome ~~ I could not handle the case where I could not respond to arbitrary requests or reuse one script according to the path, but recently a wonderful mechanism called proxy integration has been introduced. it was done. If you use this, request and path information will be passed to the function side, so the possibilities of API Gateway + Lambda will expand at once. I immediately tried it this time.

Create a method and resource for the route with API Gateway like this, and link this with Lambda to be created this time.

swagger.yaml

---
swagger: "2.0"
basePath: "/jinja"
schemes:
- "https"
paths:
paths:
  /:
    x-amazon-apigateway-any-method:
      produces:
      - "application/json"
      responses:
        200:
          description: "200 response"
          schema:
            $ref: "#/definitions/Empty"
  /{proxy+}:
    x-amazon-apigateway-any-method:
      produces:
      - "application/json"
      parameters:
      - name: "proxy"
        in: "path"
        required: true
        type: "string"
      responses: {}
definitions:
  Empty:
    type: "object"
    title: "Empty Schema"

Writing in Swagger looks like this.

By listing the resource as / {proxy +}, it is possible to handle requests of any path with the same Lambda. Since the access of the root could not be picked up only with / {proxy +}, a method is defined separately for the root as well.

Response for API Gateway

The setting on the API Gateway side was easy, but there was a problem on the Lambda side. I was wondering if I could create a Lambda that returns HTML and connect it to API Gateway, and it would return the page, but it was a little different. If you just return HTML, API Gateway will return 502. At first I was messing around with ContentType and API Gateway settings, thinking that the format wasn't JSON, but the cause was elsewhere.

When using Proxy integration, it looks like the script needs to return a response in a specific format __. It was a punch line that 502 would be returned without any questions if you did not follow this format.

Click here for the fixed format.

main.py

def handle(event, context):
    html = ...
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "text/html"},
        "body": html
    }

You need to return a dictionary that defines three things, statusCode, headers, and body, from the function called by Lambda. API Gateway looks at these and creates an HTTP response.

Also, all the information when making a request to API Gateway is passed to the handle function ʻevent` (first argument). In this sample, what kind of value is passed is displayed by pprint, so please refer to it if you like.

Complete

https://teu24wc5u9.execute-api.ap-northeast-1.amazonaws.com/jinja/ The shrine was successfully completed. I wanted to make the page a little more like a shrine, but I was exhausted because I didn't have time. I'm glad I've told you what I want to convey technically.

Can you see that the number of worshipers displayed is increasing each time they are accessed? I was able to create a dynamic web page without creating any instances.

The path to access is

/jinja/
/jinja/hoge
/jinja/fuga
/jinja/piyopiyo

Anything will be accepted. The request information path at the bottom of the page should also be properly taken. With this, it's easy to add your own routing process and show different pages.

Due to API Gateway restrictions, the / jinja / part is a version string and cannot be removed.

In order to access by root, it seems good to put CloudFront etc. in front of API Gateway. You can't host static files with API Gateway alone.

At the end

I tried to build a dynamic page with Lambda with a fairly close-out departure, but it was good to be able to make it safely. I was worried about using something for API as a web page because of the name API Gateway, but in the end it is powerful that even a full-fledged web service can be used with Lambda + API Gateway. I felt it. The only pain in Lambda is that you can only use Python2 ... Please support 3 as soon as possible: pray:

https://github.com/pistatium/lambda_jinja_sample The source for this time is here.

bonus

Our Qiita Organization Was made last time. I would like to do my best to write other than AdventCalender.