[PYTHON] Environment when AWS Lambda includes native libraries

Introduction

If you want to use native libraries in Python's dependent libraries I've heard from a long time ago that the environment to build (pip) and the environment on the AWS Lambda side are aligned.

But what is the actual effect and how is it bad? I haven't checked it at all, so It is a memo that I checked the library obtained by pip install while I was addicted to it.

Error when running Lambda

error contents

It occurred when using a module that calculates the distance called pyproj. (At first, I didn't know what was wrong with this error alone ...)

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_handler': cannot import name '_datadir' from partially initialized module 'pyproj' (most likely due to a circular import) (/var/task/pyproj/__init__.py)

Environment at the time of error

Build side: Python 3.7 Docker Image (https://hub.docker.com/_/python)
Lambda side: Python 3.8 runtime

Environment at the time of error resolution

Build side: Python 3.8 Docker Image (https://hub.docker.com/_/python)
Lambda side: Python 3.8 runtime

Contents of pyproj library (partial excerpt)

I do not understand the detailed installation mechanism, but it automatically grasps the environment and It seemed to be downloading the dependent native libraries.

Build environment is Docker Image Python 3.7

pyproj
- _crs.cpython-37m-x86_64-linux-gnu.so
- _datadir.cpython-37m-x86_64-linux-gnu.so
- proj.py

Build environment is Docker Image Python 3.8

pyproj
- _crs.cpython-38m-x86_64-linux-gnu.so
- _datadir.cpython-38m-x86_64-linux-gnu.so
- proj.py

Build environment is Windows Python 3.7 installation (bonus)

pyproj
- _crs.cp37-win32.pyd
- _datadir.cp37-win32.pyd
- proj.py

at the end

I got a little more understanding, or a sense of conviction. However, since the official Docker Image of amazon linux is distributed, it may be that the build environment should be there.