Load an external library from your Lambda function using AWS Lambda Layers. The Python environment for Amazon Linux 2 is also in place. (Python3.6, Anaconda)

Log in to the AWS Console and set up Amazon Linux 2 from EC2. The instance type is ~~ t2.micro ~~ and click the Confirm and Create button. * As will be described later, please use t2.medium. Please set the security group by yourself. Open port 22 for SSH connection. Select an existing key pair, if not, create a new one. Required for SSH connection. Creating an instance. Click View Instance. When the instance status is running and the status check is 2/2, it is complete. After creating the instance, copy the IPv4 public IP and make an SSH connection from PuTTY. Open PuTTY and enter your IP address in Host Name. Select a key pair from Connection-> SSH-> Auth. Use PuTTYgen to convert .pem to .ppk. Click Yes. You can log in with ec2-user.

After logging in, check your Python version. It seems that Python 2 series is still the default.

$ python
Python 2.7.16 (default, Dec 12 2019, 23:58:22)
[GCC 7.3.1 20180712 (Red Hat 7.3.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Install Anaconda. Copy the download URL from the Anaconda site.

Wget to the download URL and install with bash.

$ wget https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh
$ bash Anaconda3-2019.10-Linux-x86_64.sh

Read the terms with Enter and enter yes. Start installation with Enter.

Please answer 'yes' or 'no':'
>>> yes

Anaconda3 will now be installed into this location:
/home/ec2-user/anaconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

Unpacking payload ...
   0%|   | 0/291 [00:00<?, ?it/s]

It has stopped at.

It seems that you can do it with t2.medium, so stop the instance and change the instance type. current latest miniconda sh installer for linux hangs on Unpacking payload step #9345

After installing Anaconda, create a Python 3.6 environment.

$ /home/ec2-user/anaconda3/bin/conda create -n py36 python=3.6
$ /home/ec2-user/anaconda3/bin/conda init 

Reboot the instance and install the library in the python folder in the py36 environment.

$ conda activate py36
$ mkdir python
$ pip install -t ./python requests
$ pip install -t ./python ulid-py
$ pip install -t ./python pillow
$ pip install -t ./python numpy
$ pip install -t ./python opencv-python
$ pip install -t ./python opencv-contrib-python
$ pip install -t ./python pandas
$ pip install -t ./python matplotlib
$ pip install -t ./python folium

Zip the python folder.

$ zip -r GachiLayers.zip python

Install awscli settings and boto3 to upload the zip file to S3.

$ pip install awscli
$ pip install boto3

Create a user from IAM in the AWS Console to get your AWS Access Key and Secret Access Key.

Give the access authority of AmazonS3FullAccess.

There is no particular tag.

confirm.

Once created, you'll have the AWS Access Key and Secret Access Key needed for aws configure, so keep it in a safe place.

Enter the AWS Configure settings, AWS Access Key and Secret Access Key.

$ aws configure
AWS Access Key ID [None]: xxxxxxxxxxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Default region name [None]: ap-northeast-1
Default output format [None]: json

Create a Python program to upload the Zip file to S3. Create a new S3 bucket.

$ vi upload.py

import boto3

filename = 'GachiLayers.zip'
s3 = boto3.resource('s3')
obj = s3.Object(<INSERT YOUR BUCKET NAME>, filename)
response = obj.put(Body = open(filename, 'rb'))

I will do it.

$ python upload.py

Make sure it's uploaded to S3.

Open Lambda from the AWS Console and create a layer from Layers.

I was able to create it.

Create a Lambda function (runtime is Python3.6) and add a layer.

Select the created layer.

Let's edit the Lamda function to make sure the library is loaded.

import json
import requests
import ulid
from PIL import Image 
import numpy as np
import cv2
import pandas as pd 
import matplotlib.pyplot as plt
import folium
from folium import plugins

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Create a test event.

If you can, the library should be loaded.

Response:
{
  "statusCode": 200,
  "body": "\"Hello from Lambda!\""
}

Thank you for your hard work. You can delete the instance when you are done.