[PYTHON] Pip install a private repository when building Docker

Requirements

When building Docker, I want to pip install libraries from private repositories such as github and BitBucket

conditions

The key to access the private repository is under $ HOME / .ssh

Proposal 1

Create a wheel locally and copy it with Dockerfile

pip wheel djando (Although django isn't private)

good point

--Key problem and Docker can be separated --Reuse the wheel when rebuilding the image

problem

--Since the library is built in the local environment, it may not match the execution environment in the container.

Example 1: Get Django on the wheel -> Django-1.8.1-py2.py3-none-any.whl python2-3 compatible, OS independent (any) No problem in this case,

Example 2: Getting MysqlClient on wheel -> mysqlclient-1.3.10-cp27-cp27m-macosx_10_10_intel.whl CPython2.7 limits the build and execution environment for macosx10.10. Since Docker Container runs on the linux kernel, it may interfere with its operation.

It would be nice to get it on linux and convert it to a wheel, but it took too much time and it fell over.

Proposal 2

Embed the key in the image. O'REILLY's Docker book says "Don't do it". Rejected because it's too unfriendly for security.

Proposal 2-2

I don't want the key to remain in the image permanently, so I thought I should copy the key and delete it after pip install.

Dockerfile.


COPY id_rsa /root/.ssh/
COPY id_rsa.pub /root/.ssh/
RUN chmod 600 /root/.ssh/id_rsa && \
    chmod 600 /root/.ssh/id_rsa.pub && \
    echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
RUN pip install -r requirements.txt
RUN rm /root/.ssh/*

Like this. The key file access must be 600 (.ssh is 700). Also, if HostKeyChecking is not disabled, an error will occur in the build. After pip install finishes, remove all credentials from the image.

You can install it with this

Key files are usually located under $ HOME / .ssh locally, and Docker builds can only access the directory containing the Dockerfile and the files inside it as the top directory. Therefore, you need to copy the key file and bring it to an area accessible by the Docker build. It's not very cool to keep it permanently, so I think it's a good idea to write a build script that temporarily copies and deletes it after the build. It's not complete with Docker commands, but it can't be helped ...

Proposal 3

Do your best only for the library of the private repository, make it into a wheel to match the execution environment of Docker, and pip install the rest from the Internet in the build normally.

It depends on the number of libraries in the private repository, but I feel good about it. But do we also need wheel version control ...

Summary

Proposal 2-2 for distribution as a personal development environment, Proposal 3 for production operation ... (No)

Recommended Posts

Pip install a private repository when building Docker
When pip install fails
I got a UnicodeDecodeError when pip install on ubuntu
A memorandum when an error occurs with pip install
When moss with pip install
Install github repository with pip
Pip install the GitHub repository
Proxy error when running "pip install"
How to install a package using a repository
A workaround when installing pyAudio with pip.
Create a private repository with AWS CodeArtifact
Build a Docker image containing the private repository Python library on GitHub Actions
Error when installing a module with Python pip
Change the default reference repository for pip install
Pipenv install with ssh from Private Bitbucket Repository
pip install Specify the github repository as the source
Eliminates SSL error when PIP Install on Windows.
sudo pip install
pip> link> Is `sudo pip install` still a broken practice? <How about adding sudo when piping?
Building a Docker working environment for R and Python
A memo about building a Django (Python) application with Docker