Lambda and Layers are convenient, but there is a limit of 5 per function.
I think that there are many cases where Layers are operated as one module, but unexpectedly it is insufficient or it is troublesome to add one by one.
Manage your favorite modules in the familiar requirement.txt and combine them into one layer.
Create requirement.txt to import 6 modules in the test.
requirement.txt
selenium
requests
Pillow
lxml
numpy
Flask
Then execute the following script in the same layer. docker Edit the python version and aws command arguments as needed.
mkdir -p python/bin/
docker run --rm -v $(pwd):/var/task -w /var/task lambci/lambda:build-python3.7 pip install -r requirements.txt -t ./python
zip -r layer.zip python
aws lambda publish-layer-version --layer-name favorites --zip-file fileb://layer.zip --compatible-runtimes python3.7 --region ap-northeast-1
rm -rf layer.zip python
You can now add layers registered in Lambda. The test function was deployed and confirmed by from this repository Serverless Framework. that's all!
If you use Serverless Framework, serverless-python-requirements will make module management easier, and it is better not to separate using layers. I think that Infrastructure as Code is thorough and elegant. However, if the upper limit of the number of layers becomes an obstacle or the module is included in the package, the waiting time for deployment will be long, so I personally prefer to separate by layers because I hear a small turn.
By the way, if you use selenium, it is easier to manage the binaries together. Below is a script that also adds binaries. As an aside, the headless selenium that runs on python with lambda that I know is a set of this script version and python3.7. That's why you're using an older release version.
mkdir -p python/bin/
+ curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-37/stable-+ headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip
+ unzip headless-chromium.zip -d python/bin/
+ curl -SL https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip > chromedriver.zip
+ unzip chromedriver.zip -d python/bin/
+ rm -rf chromedriver.zip headless-chromium.zip
docker run --rm -v $(pwd):/var/task -w /var/task lambci/lambda:build-python3.7 pip install selenium -t ./python
zip -r layer.zip python
aws lambda publish-layer-version --layer-name selenium_with_bin --zip-file fileb://layer.zip --compatible-runtimes python3.7 --region ap-northeast-1
rm -rf layer.zip python
The test function that sets the chromeOptions for selenium to work properly on lambda and the binary path obtained from this layer is [here](https://github.com/umihico/fav-py-modules/blob/master/scraping. py).
reference https://github.com/adieuadieu/serverless-chrome/issues/133 https://dev.classmethod.jp/articles/managing-external-modules-with-serverless-framework-plugin/ https://www.npmjs.com/package/serverless-python-requirements https://hacknote.jp/archives/49974/ https://dev.classmethod.jp/articles/lambda-layer-first-action/
Recommended Posts