[LINUX] Docker Engine --CE source build


Build Docker Engine --Community Edition from source code. The latest source builds are changing in many ways. Source build information is available elsewhere, but I haven't seen any Japanese information showing the latest build instructions. If the information is a little old, the build procedure will be different [^ 1]. Therefore, the text shows the latest Docker-ce-19.03.11 source build as of June 2020.

[^ 1]: In the first place, the project structure of Docker has changed, and the Docker Engine is now under the responsibility of the Moby project. So the information you get from [https://github.com/docker/docker] for the git source is out of date. Moreover, since the Docker daemon and Docker client builds were separated at some point, the procedure in the main text is required to build the daemon and client all at once.

Please be careful if you follow the procedure described in the text. I think that the Linux distribution that many people use has a package manager and installs the Docker module as a package. The work in the text is trying to disable the packaged Docker and replace it with a self-built Docker. Also, since I am using Linux From Scratch, I do not consider the package manager, and I do not hesitate to install it as / usr / bin. Make sure you fully understand the state managed by the package manager and the risks of damaging the environment before proceeding with the text. For example, take measures such as setting the installation destination to / usr / local / bin.


The author is [Linux From Scratch](http: //) as shown in this Qiita post "Introducing Docker Engine to Linux From Scratch". I am using a Linux OS built based on www.linuxfromscratch.org), and I will build the source on that OS. However, I think this source build method works for all Linux operating systems. This is because when building from Docker source, a Docker container is created using Docker (to be exact, Docker daemon) already installed on the system (= host system) where the source build is performed, and the source is created in that container. Because it builds. Speaking like Docker, in an environment encapsulated in a container, the necessary build tools are dynamically installed, that is, built at that point, so any Linux host system can be used, and the build tools are on the host system. You can build it even if it is not in order.

In general, the following assumptions are made.


In the following, this procedure is performed by the root user. Please be careful.

1. Download source

There are two ways to get the source. One is to use the source code summarized in the tarball. And the other is to get the latest source from the git repository, as is often explained everywhere. I think you should decide which one to choose depending on your purpose and preference, but I recommend the tarball method.

1.1. Get the tarball

The tarball source is available on the official Docker Github site release page (https://github.com/docker/docker-ce/releases). There is a tarball download link in the middle of the long and long release description, which is difficult to understand, but if you look closely, you will find [Source code]. (tar.gz)](https://github.com/docker/docker-ce/archive/v19.03.11.tar.gz). In the text, the latest v19.03.11.tar.gz at the time of writing is used.

So use wget to download it. For the download directory, use / mnt / lfs / sources as an example of the directory for the source build.

It's a personal matter here, but since the tarball name is v19.03.11.tar.gz, you can't tell what the source tarball is just by looking at it. Considering that we will manage it later, we will rename the file and download it using the wget option --output-document as shown below.

# cd /mnt/lfs/sources
# wget -N https://github.com/docker/docker-ce/archive/v19.03.11.tar.gz \
    --output-document docker-ce-19.03.11.tar.gz

If you check the contents of the tarball just in case, you can see that various source codes are collected under the directory docker-ce-19.03.11.

# tar tf docker-ce-19.03.11.tar.gz | less
(The following is omitted)

Unpack (decompress) the tarball and move it to the source top directory docker-ce-19.03.11.

# tar xf docker-ce-19.03.11.tar.gz
# cd docker-ce-19.03.11

If you chose to get this tarball, skip the git description below and go to the section 2. Source Build.

1.2. Obtaining the source with git

Clone the official Docker git repository to get the latest source code. A new docker-ce directory will be created, so go to it.

# cd /mnt/lfs/sources
# git clone https://github.com/docker/docker-ce.git
# cd docker-ce

2. Source build

2.1. Source build policy decisions

Before proceeding with the source build, first set the build policy.

2.1.1. Determining the build target

If you enter the source directory and type make, this alone will not start the build process and will display a list of build targets for make. make is designed to execute with a given build target as an argument. In other words, you need to decide which make build target to choose. There are several targets, but there are three ways to build a new Docker execution module.

Unless you have a clear build purpose, the bottom line is that you should choose static. I will choose this in the text.

As the name implies, deb and rpm are supposed to build the deb and rpm packages, respectively. I haven't actually tried it, so I won't explain it in this article.

2.1.2. Measures for minimal build

Choosing the make build target static has the drawback of building too many executable modules. Specifically, it cross-compiles all execution modules for supported architectures such as Linux, Windows, Mac, and ARM. Unless you're a developer trying to create all of this, you don't need it all. Even though Docker source builds take a long time to build, building everything requires a great deal of preparedness.

My consciousness is to reconfigure the Docker daemon and client modules running on Linux OS with a source build, without the need for a Windows or Mac executable. So make sure to build only the Linux executable.

2.2. Check before build

The Docker daemon must be running in order to do a Docker source build. If it is not started, start it as follows.

# systemctl start docker
2.3. Run the build

The directory docker-ce-19.03.11 created by decompressing (decompressing) the tarball is the source top directory. (Or docker-ce is the source top directory if you got it from the git repository.)

First, change to the source top directory.

# cd /mnt/lfs/sources/docker-ce-19.03.11

Then run make static. However, if you execute it as it is, it will create all the execution modules for the corresponding architecture as mentioned above. To build only the Linux executable module, type:

# make static DOCKER_BUILD_PKGS=static-linux

I will briefly explain the meaning of the above command.

.PHONY: static
static: DOCKER_BUILD_PKGS:=static-linux cross-mac cross-win cross-arm
2.4. Verification of build processing time

By the way, I used the time command to measure the build time for all module builds by executing only make static and Linux module builds with DOCKER_BUILD_PKGS = static-linux. The build environment is the same environment of virtual image on VMware, Linux From Scratch, memory 2GB, swap 1GB. See the overwhelming difference.

Processing method Processing command processing time
All module build make static 4 hours 06 minutes 49 seconds
Build Linux module only make static DOCKER_BUILD_PKGS=static-linux 59 minutes 04 seconds

In all module builds, we are building 4 types of modules for architecture, static-linux, cross-mac, cross-win, cross-arm, and it takes about 1 hour to build one architecture. The result is that it is needed.

3. Installation (save execution module)

3.1. Confirmation of execution module

Execution modules built by make static are stored in <source top directory> / components / packaging / static / build / linux. One is a tarball of executable modules, and the other is an individual executable module in the subdirectory docker (and docker-rootless-extras). I will. The situation is as follows. You can see two tarballs and the subdirectories docker and docker-rootless-extras.

# cd /mnt/lfs/sources/docker-ce-19.03.11
# cd components/packaging/static/build/linux
# ls
docker               docker-rootless-extras
docker-19.03.11.tgz  docker-rootless-extras-19.03.11.tgz

If you continue to look at the subdirectories docker, docker-rootless-extras, you will see that the executable module is still there.

# ls docker
containerd       ctr     docker-init   dockerd
containerd-shim  docker  docker-proxy  runc

# ls docker-rootless-extras
dockerd-rootless.sh  rootlesskit  rootlesskit-docker-proxy  vpnkit

The next installation method is to copy and save the tarball somewhere, and in some cases copy it to another machine and install it. You can simply copy the executable module in the subdirectory docker (and docker-rootless-extras) directly to the host system. Choose the appropriate method according to your needs. In the text, we will proceed with the latter method.

3.2. Stop Docker service

We will reinstall the Docker service, so stop the Docker service first.

# systemctl stop docker

If you don't do this, when you try to copy and install the Docker daemon's executable module dockerd, systemd has grabbed the old dockerd, so you'll get a new executable. Overwrite copy is not possible.

3.3. Backup (or delete) of the execution module up to that point

If necessary, make a backup of the Docker module you were using. Move it to some directory and save it, or save it with a suffix such as .orig. The backup procedure is not shown in the text. Please use the method of your choice.

Alternatively, if you are well aware of the situation and it is possible, you can remove all previous Docker executables. If you have installed Docker-related packages using the package manager provided by your Linux distribution, you may want to remove them. However, I will write it many times, but only if I fully understand the situation.

3.4. Installation of execution module

As shown above, the new executable module built is placed under the source directory, so this is a method to copy it directly to / usr / bin. Simply copy from within the source directory to / usr / bin.

I explained that the single execution module is located in <sourcetop directory> / components / packaging / static / build / linux / docker. So, as before, assuming the source top directory is / mnt / lfs / sources / docker-ce-19.03.11, do the following:

# cd /mnt/lfs/sources/docker-ce-19.03.11
# cd components/packaging/static/build/linux/docker
# cp * /usr/bin

If you also want to install the rootless executable, do the same for the docker-rootless-extras directory.

# cd /mnt/lfs/sources/docker-ce-19.03.11
# cd components/packaging/static/build/linux/docker-rootless-extras
# cp * /usr/bin

Try running the docker, dockerd, and other executable modules installed in / usr / bin with the --version option. Make sure you have the new version installed.

# which -a docker

# docker --version
Docker version 19.03.11, build
# which -a dockerd

# dockerd --version
Docker version 19.03.11, build unsupported
3.5. Restart the Docker service

Start the Docker service that was stopped.

If docker.service and docker.socket are prepared correctly, a new Docker daemon execution module dockerd will start from here. I will actually move it.

# systemctl start docker.service
3.6. Operation check

After this, try docker run hello-world and Tutorial of the official Docker documentation, and the Docker daemon and Docker client will work. Make sure.

That is all for the text.

Recommended Posts

Docker Engine --CE source build
Build PostgreSQL from source
SciPy 1.4 source build requires pybind11
docker build --help Japanese translation
docker build python based on alpine
Build a go environment using Docker
Build a deb file with Docker
Build Mysql + Python environment with docker
Google App Engine development with Docker
Build PyPy execution environment with Docker