[PYTHON] [Alibaba Cloud] Do something like SSI even with OSS / Function Compute

TL;DR Last time, HTML with SSI (include virtual) described in AWS is stored in a specific bucket of S3 and the sources in the include are combined with Lambda. I made a mechanism to store it in another bucket, but it is the Alibaba Cloud version of it.

What I wanted to do

Since OSS (Object Storage Service) is a static site hosting service, it cannot be processed dynamically on the server side in principle. Of course, the CDN side (Alibaba Cloud CDN), which is the cache destination after that, cannot be used either.

In that case, I think that the standard method is to build a local development environment, keep them in separate files locally, and combine them when compiling, but OSS + is an existing site that originally used SSI. Unfortunately, that flow cannot be introduced for all projects, including when transferring to a CDN.

So, if SSI was originally used, I tried to adjust it on Alibaba Cloud side so that it can be used as it is without replacing it as it is.

What i did

(As a premise, the settings of RAM and KMS, which are the IAM of Alibaba Cloud version, have been completed)

Basically, it is customized for Alibaba Cloud based on the previous article. [AWS] Do SSI-like things with S3 / Lambda I think this article by SB Cloud is useful for setting up Alibaba Cloud. Manipulate object storage with event-driven service Fucntion Compute

Constitution

The configuration is the same as last time. So if you only use OSS and Function Compute and you need Alibaba Cloud CDN. SSI_alibaba.png

Setting method

OSS In Alibaba Cloud version, there is only one bucket to upload files. The process is divided into two directories, a directory for temp and a directory for public use.

Bucket for public / temp

The name can be anything. This time it is oss-ssi-include.

Each setting

It is assumed that the access permissions are set appropriately.

Creating a public directory and a temp directory

Create the public directory dist and the temp directory src in the bucket by clicking the "Create Directory" button.

Function Compute

Function Compute detects the PUT event, and if SSI (Server Side Includes) is described in the HTML file uploaded in the src / directory, Function Compute performs the include processing and the dist / directory. Create a function to store in.

Function code

import json
import os
import logging
import oss2 
import re
import urllib.parse

logger = logging.getLogger()
logger.setLevel(logging.INFO)

OSS_ENDPOINT        = "oss-ap-northeast-1.aliyuncs.com"
DEST_BUCKET_NAME    = "oss-ssi-include"

def handler(event, context):
    logger.info('## ENVIRONMENT VARIABLES')
    logger.info(os.environ)
    logger.info('## EVENT')
    logger.info(event)
    
    creds = context.credentials
    auth = oss2.StsAuth(creds.accessKeyId, creds.accessKeySecret, creds.securityToken)
    evt = json.loads(event)
 
    #Rewrite below
    input_bucket = oss2.Bucket(auth, OSS_ENDPOINT, DEST_BUCKET_NAME)

    logger.info('## INPUT BUKET')
    logger.info(input_bucket)
    
    input_key = urllib.parse.unquote_plus(evt['events'][0]['oss']['object']['key'])
    logger.info('## INPUT KEY')
    logger.info(input_key)

    try:
        #Get input file
        response = input_bucket.get_object(input_key)
        logger.info(response)

        #File output
        output_key    = re.sub('^src/','dist/',input_key)
        logger.info('## OUTPUT KEY')
        logger.info(output_key)

        if not input_key.endswith('.html'):
            logger.info(response)
            input_bucket.put_object(output_key, response)

        else:
            input_html = response.read().decode('utf-8')
            logger.info('## input_html')
            logger.info(input_html)
            output_html = input_html
            #Get SSI description
            include_path_base = re.findall(r'<!--#include virtual="/(.*?)" -->.*?\n', input_html, flags=re.DOTALL)
            logger.info('## PATH BASE')
            logger.info(include_path_base)
            if len(include_path_base) > 0:
                for path in include_path_base:
                    include_path = path
                    logger.info('## PATH')
                    logger.info(include_path)
            
                    #Get SSI file
                    try:
                        include = input_bucket.get_object('src/' + include_path)
                        include_html = include.read().decode('utf-8')
                        #Run SSI
                        output_html = output_html.replace('<!--#include virtual="/' + include_path + '" -->', include_html)
                    except ClientError:
                        pass
            

            input_bucket.put_object(output_key, output_html)

    except Exception as e:
        logger.info(e)
        raise e

Other settings

Trigger setting

Trigger type: OSS trigger Trigger name: put Event Source: acs:oss:ap-northeast-1:xxxxxxxxxxxxxxxxx:ossname Events: oss: ObjectCreated: PutObject, oss: ObjectCreated: PostObject Trigger rule: prefix src / Role operation: Select an existing role Existing role: The role is OK with the following policy set in RAM.

{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "fc:InvokeFunction"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

in conclusion

Last time, HTML with SSI (include virtual) described in AWS is stored in a specific bucket of S3, and the sources in the include are combined with Lambda. I made a mechanism to store it in another bucket, but I prepared it because it was necessary to make it in Alibaba Cloud for various reasons.

Unlike the AWS version, the file is uploaded to the temp directory → CloudCompute processes it → The file is stored in the public directory, so if you want to make it easier to use, it is better to deliver the file stored in the public directory to the CDN I think it's good.

Also, if you want to get information on Alibaba Cloud in Japan, SB Cloud's Engineer Blog is recommended. I also used this one as a reference.

But is this important? .. That's all from the field.

Recommended Posts

[Alibaba Cloud] Do something like SSI even with OSS / Function Compute
Explore Alibaba Cloud Function Compute for DevOps using Python 3.0
Do something fuzzing with acceptableRegex.py
I want to improve efficiency with Python even in an experimental system (3) I want to do something like Excel with Pandas
Connect your SQL Server database to Alibaba Cloud Function Compute using Python