[PYTHON] Stumble on dynamic linking when running binary in Pyinstaller (Linux) Resolved

Overview

** Pyinstaller ** https://github.com/pyinstaller/pyinstaller is a thankful library that transforms a program created with python into an executable file.

When you use this to put python code into one binary file or one folder, When I brought the binary file to another environment and tried to execute it, I got angry about dynamic linking and got stuck.

Environment & command used

environment:

pyinstaller -D --clean --onefile --additional-hooks-dir ./ main.py

Use this command to turn main.py into a onefile (binary file) I wanted to use it in other environments. (Actually, I put this binary file in docker image and tried to run it using docker)

Error


| [7] Error loading Python lib '/tmp/_MEIzvSj0a/libpython3.6m.so.1.0': dlopen: libexpat.so.1: cannot open shared object file: No such file or directory

Consideration

so file I'm angry without it. I thought that the onefile created by pyinstaller does not depend on the execution environment, Are you actually doing dynamic linking? ??

When I looked it up, I found a lot of things. First, even if it becomes one file with pyinstaller https://github.com/pyinstaller/pyinstaller/wiki/FAQ As you can see here

GNU/Linux Under Linux, I get runtime dynamic linker errors, related to libc. What should I do? The executable that PyInstaller builds is not fully static, in that it still depends on the system libc.

And that. In other words, it depends on linux because there are libraries that are dynamically referenced, of course. Here, the following was written.

Another solution is to use a tool like StaticX to create a fully-static bundled version of your PyInstaller application. StaticX bundles all dependencies, including libc and ld.so. (Python code :arrow_right: PyInstaller :arrow_right: StaticX :arrow_right: Fully-static application)

staticxHow about making it a completely static library? It was that. I tried it accordingly.

The git repository is https://github.com/JonathonReinhart/static Here it is. I used this to reduce the dynamic dependency of what became onefile with pyinstaller and further with staticx```.

As a result, it was found that the binary created here can be executed safely in the ubuntu (same as the development environment) docker environment.

So why do we need to link dynamically?

I feel that if there are many things that are dynamically linked, environment dependence will increase, but Why do you do dynamic linking?

http://archive.linux.or.jp/JF/JFdocs/libc-intro.html It describes the history of the `` `libc``` used by linux systems for operations, I will quote a little about the explanation of static linking and dynamic linking in it.

When using a static library, the linker needs a program Find the part you want to use and copy it to the output executable file. For shared libraries, the work is different. The linker says in the output file, "When this program runs, First of all, this and this library must be loaded. " Embed the message. So obviously the shared library The size of the executable file will be smaller if you use it. It also consumes less memory and less disk space. On Linux The default behavior is to use the shared library, if any, If not, it will link statically. Shared library for executables If you want to format it but it's static, put it in the right place I have a shared library file (* .sa for a.out, * .so for ELF) Check if they are readable.

In other words, it is not good to sprinkle static links so that everything is environment-independent. (That is, my thoughts were shallow.)

Postscript

When using pyinstaller to output one folder instead of one file

cannot find libexpat.so

I was getting an error like this. This says that when using libpython3.6m.so, libexpat.so``` also had to be dynamically linked, so it's not on the docker container at that time. It was an error.

I thought about it, but the quick solution is It was to install this `libexpat.so``` normally with apt-get install libexpat-dev `` on the container.

Summary

When using ** pyinstaller ** on linux, keep in mind that there are libraries that are dynamically linked even after becoming onefile. If you get angry at run time with a link issue

--You can try using staticx (but understand that static linking increases the size of the binary). --When you want to embed onefile in docker, when you create onefile on linux, it is less stressful to build docker in the same environment as possible.

that's all.

This time around here.

end.

Recommended Posts

Stumble on dynamic linking when running binary in Pyinstaller (Linux) Resolved
The day when the flask server running on linux on AWS was semi-persistent (running in the background)
Eliminate password entry when running sudo ~~ on Linux PCs!
Resolved an error when putting pygame in python3 on raspberry pi