Deploy the Docker Engine inside a handmade Linux OS built on Linux From Scratch (http://www.linuxfromscratch.org/index.html). Since Linux From Scratch is a system that is built from scratch using source code, Docker can only be built from scratch. The purpose is to understand Docker above all else. It is not aimed at putting Docker to practical use.
Since the text uses a Linux OS based on Linux From Scratch, it involves a large task of rebuilding the kernel source. If you don't intend to use Linux From Scratch, it may seem unrelated. However, after building the kernel source, the procedure for installing the Docker binary and setting the environment is shown. I think that Docker is packaged in major Linux distributions, but if you are using a distribution that does not, I think that the procedure described in the text may be helpful because you will be using Docker binaries. I have.
In order to build Docker Engine from source code, you need Docker itself. Therefore, start by installing the officially provided Docker Engine binaries.
If you follow the Linux From Scratch way of sticking to building from source code, then you'll build using Docker source code. However, creating an operating environment for Docker Engine binaries can be a daunting task. Furthermore, once that environment is created, Docker will work, so it can be seen that the purpose has been achieved for the time being. In this article, we will explain how to build an environment using Docker binaries.
Also, although not essential, for personal reasons, all steps shall be performed on a Linux From Scratch system built on VMware Workstation. The reason for this is simply that there was no Linux machine to be free. It does not contain any specific information about VMware virtual images, so you can think of it as a pure Linux operating system operation.
The PC and each software use:
| item | Contents |
|---|---|
| Laptop | Windows 7, 64bit, amd, memory 4GB (relatively poor) |
| VMware Workstation | VMware Workstaion 10.0.7 build-2844087 (quite old) |
| Linux From Scratch | LatestSVNsystemdversionSVN-20200528（linux-5.6.15、gcc-10.1.0、systemd-245), built as a virtual image on VMware. 64bit, memory 1.5GB |
| Beyond Linux From Scratch | SVNabove-Installedvarioustoolson20200528(maintoolswillbeexplainedinthefollowingprocedure) |
| Docker | Latest Docker Engine binaries: docker-19.03.10 |
If you built Linux From Scratch without thinking about it, you probably don't have enough Linux kernel settings for your Docker environment. This is because some kernel source default settings are not built-in or module build settings, and many are required for Docker environments.
To rebuild the Linux kernel, you will need to use the "host system" from which you built Linux From Scratch again. This means logging in from the "host system" again and using the chroot command to rebuild the Linux kernel in the Linux From Scratch environment.
The way to rebuild the Linux kernel is certainly to use the original "host system", but I don't think it will be necessary again to see if the "host system" can actually be prepared easily. You may have already thrown it away.
This theme is rather realistic, but quite annoying. Perhaps the quickest way is to recreate Linux From Scratch from scratch. Since the emphasis is on building a Docker environment here, detailed information on kernel rebuild is not shown. The following will focus on the kernel option settings menu during the rebuild.
The official Docker github has a script to check the required kernel settings. It's called check-config.sh. Download and run this.
# wget https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh
# sh ./check-config.sh
The output looks like this:
warning: /proc/config.gz does not exist, searching other paths for kernel config ...
info: reading kernel config from /boot/config-5.6.15 ...
Generally Necessary:
- cgroup hierarchy: nonexistent??
(see https://github.com/tianon/cgroupfs-mount)
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: missing
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: missing
- CONFIG_KEYS: enabled
- CONFIG_VETH: missing
- CONFIG_BRIDGE: missing
- CONFIG_BRIDGE_NETFILTER: missing
- CONFIG_NF_NAT_IPV4: missing
- CONFIG_IP_NF_FILTER: enabled
- CONFIG_IP_NF_TARGET_MASQUERADE: missing
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: missing
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
- CONFIG_NETFILTER_XT_MATCH_IPVS: missing
- CONFIG_IP_NF_NAT: missing
- CONFIG_NF_NAT: enabled
- CONFIG_NF_NAT_NEEDED: missing
- CONFIG_POSIX_MQUEUE: enabled
Optional Features:
- CONFIG_USER_NS: missing
- CONFIG_SECCOMP: enabled
(The following is omitted)
The kernel options listed under Generally Necessary shown here are required in your Docker environment. If it is ʻenabled, it means that it is currently enabled, and if it is missing, it means that the corresponding kernel module does not exist. By the way, if you build the Linux kernel in Linux From Scratch build with the default settings, you should see many missings. The presence of missing` does not allow Docker to work, with some exceptions.
The kernel options listed under ʻOptional Features` are literally optional and build settings as needed. Let's build it as needed in the future.
Note that the line after Generally Necessary is shown above as-cgroup hierarchy: nonexistent ??. This means that the cgroup mount does not exist or is not properly mounted. If you then configure the cgroup mount settings, this display will change as follows: We will proceed in order to obtain such an output result.
Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
Performs the process of rebuilding the Linux kernel. This means doing the Linux Kernel Build (http://www.linuxfromscratch.org/lfs/view/systemd/chapter08/kernel.html) again in the Linux From Scratch book. Then, during the procedure, from the option setting menu when make menuconfig is executed, enable the necessary options shown below and perform the Linux kernel installation.
Below are the kernel settings I made. If you do not set these at least, the Docker environment will not work. There may be mistakes or deficiencies due to the history of trial and error. As soon as I notice it, I will make corrections.
<Kernel setting items> <Option name>
General setup --->
[*] Control Group support ---> CONFIG_CGROUPS
[*] Memory controller (NEW) CONFIG_MEMCG
[*] Swap controller (NEW) CONFIG_MEMCG_SWAP
[*] Swap controller enabled by default (NEW) CONFIG_MEMCG_SWAP_ENABLED
[*] CPU controller --->
[*] Group scheduling for SCHED_OTHER CONFIG_CGROUP_SCHED
[*] CPU bandwidth provisioning for FAIR_GROUP_SCHED CONFIG_CFS_BANDWIDTH
[*] IO controller (NEW) CONFIG_BLK_CGROUP
[*] PIDs controller (NEW) CONFIG_CGROUP_PIDS
[*] Device controller (NEW) CONFIG_CGROUP_DEVICE
[*] Networking support ---> CONFIG_NET
Networking options --->
[*] Network packet filtering framework (Netfilter) ---> CONFIG_NETFILTER
[*] Advanced netfilter configuration CONFIG_NETFILTER_ADVANCED
<M> Bridged IP/ARP packets filtering (NEW) CONFIG_BRIDGE_NETFILTER
Core Netfilter Configuration --->
<M> LOG target support CONFIG_NETFILTER_XT_TARGET_LOG
*** Xtables matches ***
<M> "addrtype" address type match support CONFIG_NETFILTER_XT_MATCH_ADDRTYPE
IP: Netfilter Configuration --->
<*> Packet filtering CONFIG_IP_NF_FILTER
<M> iptables NAT support CONFIG_IP_NF_NAT
<M> MASQUERADE target support (NEW) CONFIG_IP_NF_TARGET_MASQUERADE
<M> 802.1d Ethernet Bridging CONFIG_BRIDGE
Device Drivers --->
[*] Network device support --->
[*] Network core driver support CONFIG_NET_CORE
<*> Virtual ethernet pair device CONFIG_VETH
File systems --->
<*> Overlay filesystem support CONFIG_OVERLAY_FS
(Linux-5.6.15 Kernel setting menu)
I don't really care about the built-in setting "* "and the module build setting" M ". If possible, select "*" for those that can be built in, and select " M "for those that cannot.
After completing the Linux kernel build, reboot the system based on this new Linux kernel. Then try running the above check-config.sh again. Most of the output should be ʻenabled instead of missing`.
The system requirements for deploying Docker binaries in the first place are described in the Prerequisites section of the official documentation Install Docker Engine from binaries. The following is a simple translation of it.
Even systems that have just been built with Linux From Scratch already have procps and xz installed. For 64-bit and kernel 3.10 and above, you have no choice but to choose it. All you need here is git and iptables. The cgroup hierarchy is a big theme, so I'll explain it later.
Before we start installing git and ʻiptables, we'll install make-ca`, which is probably a must. The system requirements shown above may be too obvious and unspecified. For Linux From Scratch, which creates everything from scratch, there is no software that can be obtained without building and installing it yourself.
P11-kit is required to include make-ca, and wget is required during the installation of make-ca. Since wget is used for something, we will introduce it here. make-ca, p11-kit, and wget are all shown in the Beyond Linux From Scratch section. Follow the installation instructions provided for installation.
2.2. git
Git in Beyond Linux From Scratch Follow the git.html) page to install. Dependent packages should be cURL. The version is the latest git-2.27.0 at the time of writing the text.
2.3. iptables Iptables in Beyond Linux From Scratch Follow the iptables.html) page to install. Dependent packages are not included (because it is troublesome). The version is ʻiptables-1.8.4`, which is the latest version at the time of writing.
The iptables configuration file / etc / systemd / scripts / iptables uses the settings shown in the Personal Firewall section on the above page. Add settings as needed.
On the Install static binaries (https://docs.docker.com/engine/install/binaries/) page of the Docker documentation, you will find the binary download URL https://download.docker.com/linux/static/stable/ It is shown. Since subdirectories for each architecture are prepared, here [docker-19.03.10.tgz] under x86_64 (https://download.docker.com/linux/static/stable/x86_64/docker-19.03. Download 10.tgz). All subsequent operations are done as the root user. (Reference: Official document Install static binaries)
# wget https://download.docker.com/linux/static/stable/x86_64/docker-19.03.10.tgz
This tarball is archived with binary modules in a directory called docker.
# tar tf docker-19.03.10.tgz
docker/
docker/ctr
docker/docker
docker/containerd
docker/docker-proxy
docker/runc
docker/docker-init
docker/containerd-shim
docker/dockerd
Therefore, after decompressing (decompressing) the tarball, install it in / usr / bin as shown below. (Reference: Official document Install static binaries)
# tar xf docker-19.03.10.tgz
# cp -p docker/* /usr/bin
The above binary tarball does not include the service configuration files docker.service, docker.socket for systemd. So get them from the official github and move them to the directory / lib / systemd / system that contains the systemd services. (Reference: Official document Control Docker with systemd> Manually create the systemd unit files)
# wget https://raw.githubusercontent.com/moby/moby/master/contrib/init/systemd/docker.service
# wget https://raw.githubusercontent.com/moby/moby/master/contrib/init/systemd/docker.socket
# cp docker.{service,socket} /lib/systemd/system
dockerCreate group docker. (Reference: Official document Post-installation steps for Linux)
# groupadd docker
systemd 243 or later? In addition, it seems that cgroup will be adopted as v2 by default, and it seems that various functions of Docker will not work if it is left as v2. (I forgot the source of the information I got, so I will add it later)
To solve this, add systemd.unified_cgroup_hierarchy = 0 as a boot option for /boot/grub/grub.cfg.
systemd.unified_cgroup_hierarchy=0
Other sites have shown a solution everywhere to add this to GRUB_CMDLINE_LINUX_DEFAULT and run ʻupdated-grub, but the same is true. Since Linux From Scratch does not use GRUB_CMDLINE_LINUX_DEFAULT, specify it directly to grub.cfg`.
As a concrete example, the author's /boot/grub/grub.cfg is as follows.
# cat /boot/grub/grub.cfg
# Begin /boot/grub/grub.cfg
set default=0
set timeout=5
insmod ext2
set root=(hd0,1)
menuentry "GNU/Linux, Linux 5.6.15-lfs-20200528-systemd" {
linux /vmlinuz-5.6.15-lfs-20200528-systemd root=/dev/sda3 ro net.ifnames=0 biosdevname=0 systemd.unified_cgroup_hierarchy=0
}
I've added systemd.unified_cgroup_hierarchy = 0 to the end of the linux line. By the way, since I am using / boot mounted on a dedicated partition, the above linux line should be written as linux / vmlinux ... instead of linux / boot / vmlinux .... I have.
Reboot the system after the fix.
cgroup mountSet the mount for cgroup (control group). Add the following description to the / etc / fstab file and mount it. (Reference: Docker's Github issue # 2683)
# echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab
# mount -a
As a confirmation, try the following: This is the first check to see if the kernel module has been generated.
# less /proc/cgroups
#subsys_name hierarchy num_cgroups enabled
cpuset 4 1 1
cpu 3 1 1
cpuacct 3 1 1
blkio 5 1 1
memory 2 42 1
devices 6 29 1
freezer 7 1 1
The output should look something like the one above. The important thing here is that there are lines for memory and devices. If you don't see this, it means that the Linux kernel options mentioned above have not been built properly.
Then do the following: This is a check to see if the cgroup is properly mounted.
# mount | grep ^cgroup
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
You can see that each group (such as cpu and devices) exists and is mounted under / sys / fs / cgroup.
First, check that the Docker daemon has started.
It's okay to run the systemd service docker.service, but it's more convenient to run dockerd directly because it will give you an error message directly. Execute the dockerd command as follows. (Reference: Official document Install static binaries)
# dockerd &
If an error occurs, you will see a message similar to the following on the last line of the output:
failed to start daemon: Error XXXXXXXXXX
If there is no such error message, it seems that it started normally for the time being.
docker infoTry running docker info.
# docker info
I got the following output:
Client:
Debug Mode: false
Server:
Containers: 5
Running: 0
Paused: 0
Stopped: 5
Images: 0
Server Version: 19.03.10
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 5.6.15
Operating System: Linux From Scratch 20200528-systemd
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.432GiB
Name: DockerOnLFS
ID: PMIG:ONBA:YGH6:XMWF:GFGH:EW66:2GBR:A2E4:4D5X:TBWT:M2BL:KRVE
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
docker run hello-worldTry running docker run hello-world. (Reference: Official document Install static binaries)
# docker run hello-world
I got the following output:
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:6a65f928fb91fcfbc963f7aa6d57c8eeb426ad9a20c7ee045538ef34847f44f1
Status: Downloaded newer image for hello-world:latest
INFO[2020-06-08T10:56:23.077433898+09:00] shim containerd-shim started
address="/containerd-shim/moby/fa9173d0b26b33c1075296abd578df5c0c088a61f5683214d60bbf5d1f12fa47/shim.sock" debug=false pid=504
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
INFO[2020-06-08T10:56:24.864625786+09:00] shim reaped
id=fa9173d0b26b33c1075296abd578df5c0c088a61f5683214d60bbf5d1f12fa47
INFO[2020-06-08T10:56:24.874963601+09:00] ignoring event
module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
After this, I tried running the official documentation Tutorial part2. It's working fine.
dockerd processKill the dockerd process you ran in the above steps.
# kill -SIGKILL $(pidof dockerd)
docker.serviceStart docker.service. (Reference: Official document Control Docker with systemd> Start the Docker daemon)
# systemctl start docker.service
If there are no errors, there should be no errors and no output.
If necessary, run systemctl enable docker.service so that this service will start automatically when the system boots. (Reference: Official document [Post-installation steps for Linux> Configure Docker to start on boot](https://docs.docker.com/engine/install/linux-postinstall/#configure-docker-to-start-on- boot)))
# systemctl enable docker.service
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Check the status of the service just in case.
# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset:
enabled)
Active: active (running) since Mon 2020-06-08 11:27:52 JST; 9min ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 269 (dockerd)
Tasks: 27 (limit: 1755)
Memory: 98.5M
CGroup: /system.slice/docker.service
tq269 /usr/bin/dockerd -H fd://
mq284 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
June 08 11:27:50 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:50.415279942+09:00" level=warning msg="Your kernel does not support cgroup blkio throttle.write_bps_device"
June 08 11:27:50 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:50.415333378+09:00" level=warning msg="Your kernel does not support cgroup blkio throttle.read_iops_device"
June 08 11:27:50 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:50.415385780+09:00" level=warning msg="Your kernel does not support cgroup blkio throttle.write_iops_device"
June 08 11:27:50 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:50.438407022+09:00" level=info msg="Loading containers: start."
June 08 11:27:51 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:51.864165833+09:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
June 08 11:27:51 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:51.957543814+09:00" level=info msg="Loading containers: done."
June 08 11:27:52 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:52.591932252+09:00" level=info msg="Docker daemon" commit=9424aea graphdriver(s)=overlay2 version=19.03.10
June 08 11:27:52 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:52.663437436+09:00" level=info msg="Daemon has completed initialization"
June 08 11:27:52 DockerOnLFS systemd[1]: Started Docker Application Container Engine.
June 08 11:27:52 DockerOnLFS dockerd[269]: time="2020-06-08T11:27:52.950638271+09:00" level=info msg="API listen on /run/docker.sock"
By the way, DockerOnLFS is the host name.
Also, warning messages such as Your kernel does not support cgroup blkio throttle.write_bps_device are displayed because the corresponding kernel module has not been built. At the moment I don't understand how Docker works and what those modules are, so I'll understand it in the future and just ignore it for now.
That is all for the text.
Recommended Posts