[AWS lambda] Deploy including various libraries with lambda (generate a zip with a password and upload it to s3) @ Python

Thing you want to do

As it is, deploy it including the library such as lbxxx.so, generate a zip with a password on AWS lambda, and store it in the S3 bucket.

The wall that stands up

The wall stands in front of what I want to do.

  1. ** zipfile (module for creating zip files provided as standard in Python) has a password Zip file encryption is not supported! ** **
  2. ** You can't create a password-protected zip just by zipping the local source code into a zip file and deploying it to lambda! ** **

I would like to leave here how I got over this wall.


Local development environment

It was developed conveniently using the "Remote-Containers" function of VS Code. Please refer to this article for Remote-Containers.

AWS environment

I don't have to write anything in particular ...

Before hitting the wall

    └ .devcontainer/
        ├ Dockerfile
        └ devcontainer.json
    └ package/
        └  lambda_hundler.py
    └ requirements.txt

For Dockerfile and devcontainer.json, I got it from source code here. lambda_hundler.py is as follows.


#It is the source code of lambda
import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')

How to get over wall ①

Let's get started when we're ready. Why can't the zipfile module generate a password-protected zip file! (Abrupt) After investigating, it seems that ** pyminizip ** can generate a zip file with a password. Let's check it immediately.


pyminizip requires zlib (a free library for compressing and decompressing data), so install it.

$ sudo apt install zlib
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package zlib

** I was angry. ** ** Use "zlib1g" instead.

$ sudo apt install zlib1g

Put the following in requirements.txt and type the pip install command.


$ cd /workspaces/zip-sample
$ pip install -r requirements.txt -t ./package

Add the following to lambda_hundler.py.


import json
import os
import pyminizip

def lambda_handler(event, context):
    #We will use the tmp directory in consideration of uploading it to lambda later.
    zip_path = "/tmp/zip/"

    # /tmp/If the zip directory does not exist, create one.
    if not os.path.isdir(zip_path):

    KEY = "/tmp/hello.txt"
    with open(KEY, mode='w') as f:
        f.write('this is test.')

    password = "password"
    compression_level = 9 #Compression level 1-9, the larger the compression, the stronger
    #The first argument is an array of file paths to include in the zip file
    #The second argument is the hierarchy in the zip file
    #The third argument is the location and file name of the zip file.
    #The fourth argument is the password
    #The fifth argument is the compression level
    pyminizip.compress_multiple([KEY], ["\\"], "/tmp/zip/sample.zip", password, compression_level)

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')

#The following is excluded at the time of deployment.

Let's start it below.

$ python lambda_hundler.py
$ ls /tmp/zip

You've done it safely!

How to get over wall ②

Now that you have successfully generated a password-protected zip file locally (docker container), let's deploy it to lambda.

$ cd /workspaces/zip-sample/package
$ zip -r ../function.zip .
$ aws lambda update-function-code --function-name sample-zip --zip-file fileb://../function.zip

After the deployment is completed, let's execute "test" from the AWS Management Console --lambda --sample-zip.

[ERROR] OSError: error in closing \tmp\zip\sample.zip (-102)
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 78, in lambda_handler
    pyminizip.compress_multiple([KEY], ["\\"], r"\tmp\zip\sample.zip", "password03", 1)

I got scolded

After a lot of research, it seems that this is happening because ** zlib is not included in the deployment package ... **

The battle to include zlib in the deployment package begins ...!

If you type the following command, it will be included in the deployment package.

$ wget http://www.zlib.net/zlib-1.2.11.tar.gz
$ tar -xvzf zlib-1.2.11.tar.gz
$ cd zlib-1.2.11
$ ./configure --prefix=/workspaces/sample-zip
share lib include/workspaces/sample-zip/Should be able to
$ sudo make install

$ cd /workspaces/sample-zip/
Include share lib include in zip file
$ zip -gr function.zip lambda_function.py share lib include
$ aws lambda update-function-code --function-name sample-zip --zip-file fileb://function.zip

As for the linux library, once you do the above, you do not have to change it, so after that you only need to zip the source code.

$ cd /workspaces/zip-sample/package
$ zip -r ../function.zip .
$ aws lambda update-function-code --function-name sample-zip --zip-file fileb://../function.zip

After the deployment is completed, let's execute "test" from the AWS Management Console --lambda --sample-zip. **success! ** (should)

Complete system

The source code after overcoming the wall is as follows.


def lambda_handler(event, context):
    zip_path = "/tmp/zip/"

    if not os.path.isdir(zip_path):

    KEY = '/tmp/hello.txt'
    with open(KEY, mode='w') as f:
        f.write("this is test.")

    password = "password"
    compression_level = 9 #Compression level 1-9, the larger the compression, the stronger
    pyminizip.compress_multiple([KEY], ["\\"], "/tmp/zip/sample.zip", password, compression_level)
    #Upload the created zip file to s3
    s3 = boto3.resource('s3')
    s3.Bucket(BUCKET).upload_file(Filename="/tmp/zip/sample.zip", Key="sample.zip")

    return {
        'status': 200,
        'body': 'Processing is finished'


--If you want to create a password-protected zip file in Python, "pyminizip"! --When deploying to lambda, include "zlib" in the deploy package!

Articles that I referred to

Recommended Posts

[AWS lambda] Deploy including various libraries with lambda (generate a zip with a password and upload it to s3) @ Python
Quickly create a Python data analysis dashboard with Streamlit and deploy it to AWS
Connect to s3 with AWS Lambda Python
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3
Upload what you got in request to S3 with AWS Lambda Python
Upload data to s3 of aws with a command and update it, and delete the used data (on the way)
I tried to automatically generate a password with Python3
[Python] Generate a password with Slackbot
[Python] Convert CSV file uploaded to S3 to JSON file with AWS Lambda
[AWS] Link Lambda and S3 with boto3
[AWS] Create a Python Lambda environment with CodeStar and do Hello World
Deploy a Python app on Google App Engine and integrate it with GitHub
Set up a Lambda function and let it work with S3 events!
How to generate a QR code and barcode in Python and read it normally or in real time with OpenCV
Export RDS snapshot to S3 with Lambda (Python)
Upload files to Google Drive with Lambda (Python)
A story about cross-compiling a python package for AWS Lambda and deploying it serverless
[Introduction to system trading] I drew a Stochastic Oscillator with python and played with it ♬
Try running a Schedule to start and stop an instance on AWS Lambda (Python)
Deploy Python3 function with Serverless Framework on AWS Lambda
Create a Layer for AWS Lambda Python with Docker
I want to AWS Lambda with Python on Mac!
Make ordinary tweets fleet-like with AWS Lambda and Python
I tried uploading / downloading a file to AWS S3 / Azure BlobStorage / GCP CloudStorage with Python
Try to generate a cyclic peptide from an amino acid sequence with Python and RDKit
Recursively get the Excel list in a specific folder with python and write it to Excel.
I made a server with Python socket and ssl and tried to access it from a browser
[Python] What is a tuple? Explains how to use without tuples and how to use it with examples.
I wrote a script to create a Twitter Bot development environment quickly with AWS Lambda + Python 2.7
Associate Python Enum with a function and make it Callable
[AWS] Try adding Python library to Layer with SAM + Lambda (Python)
Generate a password that is easy to remember with apg
Output CloudWatch Logs to S3 with AWS Lambda (Pythyon ver)
Try to bring up a subwindow with PyQt5 and Python
Create a deploy script with fabric and cuisine and reuse it
[Python] Regularly export from CloudWatch Logs to S3 with Lambda
Site monitoring and alert notification with AWS Lambda + Python + Slack
I wrote AWS Lambda, and I was a little addicted to the default value of Python arguments
Find the white Christmas rate by prefecture with Python and map it to a map of Japan
Create Cognito user list in S3 with SQS Deploy queue function and API to Lambda with SAM
Move CloudWatch logs to S3 on a regular basis with Lambda
Create a temporary file with django as a zip file and return it
[Python 3.8 ~] How to define a recursive function smartly with a lambda expression
Send images taken with ESP32-WROOM-32 to AWS (API Gateway → Lambda → S3)
How to make a surveillance camera (Security Camera) with Opencv and Python
I tried to make a periodical process with Selenium and Python
Make a scraping app with Python + Django + AWS and change jobs
Prepare an environment to use OpenCV and Pillow with AWS Lambda
Install pip in Serverless Framework and AWS Lambda with Python environment
[Python] I wrote a REST API using AWS API Gateway and Lambda.
[For Python] Quickly create an upload file to AWS Lambda Layer
How to create a serverless machine learning API with AWS Lambda
I ran GhostScript with python, split the PDF into pages, and converted it to a JPEG image.
Use AWS lambda to scrape the news and notify LINE of updates on a regular basis [python]
Notify HipChat with AWS Lambda (Python)
ImportError when trying to use gcloud package with AWS Lambda Python version
How to generate a new loggroup in CloudWatch using python within Lambda
2. Make a decision tree from 0 with Python and understand it (2. Python program basics)
I want to do it with Python lambda Django, but I will stop
Build your Django app on Docker and deploy it to AWS Fargate
Easy server monitoring with AWS Lambda (Python) and result notification in Slack