Introduction

Following Building an Apt cache server on QNAP Build a Pypi cache server as a Docker container on the NAS of QNAP. We aim to speed up pip and reduce traffic in the LAN.

Pypi cache server

Devpi server is used as the cache server for Pipit. Since it is operated by Docker, search for the devpi image. For example, scrapinghub / docker-devpi was found.

As mentioned in Last time, the NAS at hand was ARM-based. Therefore, a devpi image for ARM was created separately. The Dockerfile of the created image is GitHub, and the image is DockerHub. It is located in / r / jkawamoto / armhf-devpi-server /).

start devpi server container

Container Station is used to manage containers. If not, install it from the App Center. You can refer to the Container Station manual from here.

Search for the container image to be used from the container creation tab. By default, the image on DockerHub is available.

When you find the desired image, install it.

If the installation is complete or already installed The container creation screen opens.

You can choose the name of the container as you like. Commands and entry points can override the Dockerfile settings here, The default is fine unless otherwise instructed. CPU and memory limits vary depending on the environment, so leave the defaults here. Next, check the network items in the detailed settings.

Check which port is forwarded in the network settings. devpi server uses 3141 by default. Since the port number on the host side is auto at the beginning, change it to 3141. Also, depending on the base image, unrelated ports are open, so delete them. In the case of the figure below, 6080 is unnecessary, so delete it.

You can also mount the directory used by the cache server in the shared folder item. After completing the above settings, create a container.

Settings on the cache server user side

To use the cache server for the pip command --index-url Optional cache server URL, Pass the IP address of the cache server as the --trusted-host option. trusted-host is needed because the cache server only listens on http, not https. It's a hassle to support https, so I decided to use it only within a trusted network. Set trusted-host.

Note that the URL of the cache server is based on the IP address xxx.xxx.xxx.xxx. It becomes http://xxx.xxx.xxx.xxx:3141/root/pypi. Note that the / root / pypi part is easy to forget. (See Documentation for URL addresses other than / root / pypi)

However, it is troublesome to add options to the pip command every time, so It can also be used as a configuration file. Describe the following contents in ~ / .pip / pip.conf.

`pip.conf`


[global]
index-url="http://xxx.xxx.xxx.xxx:3141/root/pypi"
trusted-host="xxx.xxx.xxx.xxx"

After that, when using pip, you will access the cache server.

Use with Docker

The most common use of pip in Docker is at build time, that is, in Dockerfile, rather than at container execution time. Use the ARG command to decide whether to use the cache server during docker build.

`Dockerfile`


ARG PIP_PROXY
RUN if [ -n "$PIP_PROXY" ]; then \
      echo "Set pip proxy: $PIP_PROXY"; \
      IPPORT=${PIP_PROXY#*//}; \
      mkdir -p ~/.pip/; \
      echo "[global]\nindex-url=$PIP_PROXY/root/pypi\ntrusted-host=${IPPORT%:*}" >> ~/.pip/pip.conf; \
      cat ~/.pip/pip.conf; \
    fi

Add the above items to the Dockerfile. Variables are defined in ʻARG, and values can be passed with the —build-argoption duringdocker build`.

For the Dockerfile above, enter --build-arg PIP_PROXY = http: //xxx.xxx.xxx.xxx:3141/ If you pass it to docker build, it will set ~ / .pip / pip.conf appropriately. If you do not set the PIP_PROXY variable using —build-arg, it will be skipped.

From the above, it is possible to decide the use of the cache server at build time.

Summary

Prepare a devpi server container using the Container Station of the QNAP NAS. did. People in the ARM environment take time, so it will be easier to use the image created this time. Also, the ARG command of Dockerfile is used to switch the use of the cache server at build time.

This time as well, (with Docker) the story of building a PyPI cache server and making me a little happy again was helpful.

[PYTHON] Build a Pypi cache server on QNAP