Load an external library from your Lambda function using AWS Lambda Layers. The Python environment for Amazon Linux 2 is also in place. (Python3.6, Anaconda)

Log in to the AWS Console and set up Amazon Linux 2 from EC2. image.png The instance type is ~~ t2.micro ~~ and click the Confirm and Create button. * As will be described later, please use t2.medium. image.png Please set the security group by yourself. Open port 22 for SSH connection. image.png Select an existing key pair, if not, create a new one. Required for SSH connection. image.png Creating an instance. Click View Instance. image.png When the instance status is running and the status check is 2/2, it is complete. image.png After creating the instance, copy the IPv4 public IP and make an SSH connection from PuTTY. image.png Open PuTTY and enter your IP address in Host Name. image.png Select a key pair from Connection-> SSH-> Auth. Use PuTTYgen to convert .pem to .ppk. image.png Click Yes. image.png You can log in with ec2-user. image.png

After logging in, check your Python version. It seems that Python 2 series is still the default.

$ python
Python 2.7.16 (default, Dec 12 2019, 23:58:22)
[GCC 7.3.1 20180712 (Red Hat 7.3.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Install Anaconda. Copy the download URL from the Anaconda site. image.png

Wget to the download URL and install with bash.

$ wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh
$ bash Anaconda3-2019.10-Linux-x86_64.sh

Read the terms with Enter and enter yes. Start installation with Enter.

Please answer 'yes' or 'no':'
>>> yes

Anaconda3 will now be installed into this location:
/home/ec2-user/anaconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below
Unpacking payload ...
   0%|   | 0/291 [00:00<?, ?it/s]

It has stopped at.

It seems that you can do it with t2.medium, so stop the instance and change the instance type. current latest miniconda sh installer for linux hangs on Unpacking payload step #9345

After installing Anaconda, create a Python 3.6 environment.

$ /home/ec2-user/anaconda3/bin/conda create -n py36 python=3.6
$ /home/ec2-user/anaconda3/bin/conda init 

Reboot the instance and install the library in the python folder in the py36 environment.

$ conda activate py36
$ mkdir python
$ pip install -t ./python requests
$ pip install -t ./python ulid-py
$ pip install -t ./python pillow
$ pip install -t ./python numpy
$ pip install -t ./python opencv-python
$ pip install -t ./python opencv-contrib-python
$ pip install -t ./python pandas
$ pip install -t ./python matplotlib
$ pip install -t ./python folium

Zip the python folder.

$ zip -r GachiLayers.zip python

Install awscli settings and boto3 to upload the zip file to S3.

$ pip install awscli
$ pip install boto3

Create a user from IAM in the AWS Console to get your AWS Access Key and Secret Access Key. image.png

Give the access authority of AmazonS3FullAccess. image.png

There is no particular tag. image.png

confirm. image.png

Once created, you'll have the AWS Access Key and Secret Access Key needed for aws configure, so keep it in a safe place. image.png

Enter the AWS Configure settings, AWS Access Key and Secret Access Key.

$ aws configure
AWS Access Key ID [None]: xxxxxxxxxxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Default region name [None]: ap-northeast-1
Default output format [None]: json

Create a Python program to upload the Zip file to S3. Create a new S3 bucket.

$ vi upload.py
import boto3

filename = 'GachiLayers.zip'
s3 = boto3.resource('s3')
obj = s3.Object(<INSERT YOUR BUCKET NAME>, filename)
response = obj.put(Body = open(filename, 'rb'))

I will do it.

$ python upload.py

Make sure it's uploaded to S3. image.png

Open Lambda from the AWS Console and create a layer from Layers. image.png

I was able to create it. image.png

Create a Lambda function (runtime is Python3.6) and add a layer. image.png

Select the created layer. image.png

Let's edit the Lamda function to make sure the library is loaded.

import json
import requests
import ulid
from PIL import Image 
import numpy as np
import cv2
import pandas as pd 
import matplotlib.pyplot as plt
import folium
from folium import plugins

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Create a test event. image.png

If you can, the library should be loaded.

Response:
{
  "statusCode": 200,
  "body": "\"Hello from Lambda!\""
}

Thank you for your hard work. You can delete the instance when you are done.

Recommended Posts

Load an external library from your Lambda function using AWS Lambda Layers. The Python environment for Amazon Linux 2 is also in place. (Python3.6, Anaconda)
Feel free to turn Python using the library into an AWS Lambda function
Securely deploy your Lambda function using Python built with the same options as Amazon Linux
[AWS / Lambda] How to load Python external library
View images in OpenCV from Python using an external USB camera on your MacBook
Operate Maya from an external Python interpreter using the rpyc module
Write AWS Lambda function in Python
Python in is also an operator
Notify using Notification Center when the execution environment is macOS in Python
What is "mahjong" in the Python library? ??
Notes from installing Homebrew to building an Anaconda environment for Python with pyenv
[AWS] Let's run a unit test of Lambda function in the local environment
[AWS Lambda] Create a deployment package using the Docker image of Amazon Linux
[AWS] Addressing an issue where Lambda called from CodePipeline is still in progress