[PYTHON] A quick explanation from creating AWS Lambda Layers to linking

Introduction

This is a premise article dealing with Python.

What are Layers?

According to the guide

A layer is a ZIP archive that contains libraries, custom runtimes, or other dependencies. You can use a library of functions by using layers.

Layering commonly used modules such as libraries eliminates the need to include libraries in the deployment package. In other words, when using a third-party library or module with Lambda, you can store it in the layer and call it from each Lambda function.

Note

・ Up to 5 layers can be used at one time on Lambda -Deployment package size limit after decompression 250 MB

Directory structure

The path of the Lambda execution environment is the / opt directory, so The directory structure of Layer is / opt / python / Layer.

The package structure of the layer itself looks like this.

python
├ layer.py (Code such as common processing can also be entered. Can be handled in common)
├ Crypto (External library part 1)
├ psycopg2 (External library part 2)
├ sqlalchemy (External library part 3)
…etc

Creating a layer

Let's create it locally.

Pip install the library by location.

$ pip install pycryptodome -t .

And zip

$ zip -r  ../python .

Create a layer on AWS

From top to bottom ・ Name (required) ・ Explanation (not required) ・ Upload (select the zip created earlier) -Compatible runtime option (note that it is easy to forget) ・ License-Option (not required)

Click Create to create the layer.

Link to Lambda function

Now associate the Layer with the Lambda function.

Select "Designer" from the Lambda function to which you want to associate the Layer, Select "Add Layer".

Click "Custom Layer" from "Select Layer" and select the created layer. Then set the "Version" and click "Add" to complete.

When calling from Lambda, import and call.

import layer

Common mistakes

・ Lambda runs on Amazon Linux, so Some libraries may cause an execution error depending on the environment in which pip install is performed.

-Layer uploaded to AWS can also be downloaded for each version. And when the number of layers increases, I think that it may be managed like this in the directory.

test_layer
├ python
    ├ layer.py 
    ├ Crypto 

In this case, it tends to be unzipped as it is, modified and zipped, If you zip it as it is, the parent directory will also be compressed and it will not match the directory hierarchy when expanding after uploading. Zip the contents as you did when you created it.

$ zip -r  ../python .

Summary

Layer is very useful for sharing code between Lambda if used well. Let's make good use of it.

reference

https://docs.aws.amazon.com/ja_jp/lambda/latest/dg/configuration-layers.html

https://stackoverflow.com/questions/54467095/module-initialization-error-cannot-load-native-module-crypto-cipher-raw-ecb