[LINUX] Check out how Docker works with bpftrace

Overview

Docker is often used like a Black Box with docker run or docker exec commands, but I'd like to investigate its contents in more detail. One of the leading tools for that is bpftrace.

Work environment

Docker configuration

Docker is composed of the following parts

What Docker is doing

We all know that Container can be enabled by using OS functions such as Namespace, Cgroup, OverlayFS, and Virtual Network. How do you use Docker with these functions?

First, let's check the result of the Container. Start this easiest Container

$ docker run -ti --rm --name test alpine sh

Save the pid and Container ID in for future use

$ export CPID=$(docker inspect test -f '{{ .State.Pid }}')
$ echo $CPID
230973
$ export CID=$(docker inspect test -f '{{ .Id }}')
$ echo $CID
b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0

Parent-child relationship with containerd

$ pstree -s $CPID -STUpau
systemd,1
  └─containerd,80405
      └─containerd-shim,230955 -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd ...
          └─sh,230973,ipc,mnt,net,pid,uts

containerd runs containerd-shim, and also runs the Container process. Also, this Container process has namespaces such as ʻipc and mnt`.

runc settings

Since containerd uses runc, check its setting as well.

$ jq -r '.process' < /var/run/containerd/io.containerd.runtime.v1.linux/moby/$CID/config.json
{
  "terminal": true,
  "user": {
    "uid": 0,
    "gid": 0,
　　...
  },
  "args": [
    "sh"
  ],
  "env": [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "HOSTNAME=b6431f14cf40",
    "TERM=xterm"
  ],
  "cwd": "/",
  "capabilities": {
    "bounding": [
      "CAP_CHOWN",
      "CAP_DAC_OVERRIDE",
      "CAP_FSETID",
      "CAP_FOWNER",
      "CAP_MKNOD",
      "CAP_NET_RAW",
      "CAP_SETGID",
      "CAP_SETUID",
      "CAP_SETFCAP",
      "CAP_SETPCAP",
      "CAP_NET_BIND_SERVICE",
      "CAP_SYS_CHROOT",
      "CAP_KILL",
      "CAP_AUDIT_WRITE"
    ],
    ...
  },
  "apparmorProfile": "docker-default",
  "oomScoreAdj": 0
}

Namespaces

Let's take a look at the namespace attached to the Container process.

$ lsns -p $CPID --output-all
        NS TYPE   PATH                NPROCS    PID   PPID COMMAND    UID USER NETNSID NSFS
4026531835 cgroup /proc/1/ns/cgroup      120      1      0 /sbin/init   0 root
4026531837 user   /proc/1/ns/user        120      1      0 /sbin/init   0 root
4026532186 mnt    /proc/230973/ns/mnt      1 230973 230955 sh           0 root
4026532187 uts    /proc/230973/ns/uts      1 230973 230955 sh           0 root
4026532188 ipc    /proc/230973/ns/ipc      1 230973 230955 sh           0 root
4026532189 pid    /proc/230973/ns/pid      1 230973 230955 sh           0 root
4026532191 net    /proc/230973/ns/net      1 230973 230955 sh           0 root       0 /run/docker/netns/904e1ae9696c

Cgroups

Click here for the Cgroup you belong to

$ cat /proc/$CPID/cgroup
12:freezer:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
11:devices:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
10:blkio:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
9:rdma:/
8:pids:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
7:memory:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
6:hugetlb:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
5:net_cls,net_prio:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
4:cpu,cpuacct:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
3:cpuset:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
2:perf_event:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
1:name=systemd:/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0
0::/system.slice/containerd.service

And let's take a look at the contents of the Cgroup

$ cgget -g pids:/docker/$CID
/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0:
pids.current: 1
pids.events: max 0
pids.max: max
$ cgget -g net_cls,net_prio:/docker/$CID
/docker/b6431f14cf40267d3eed22b34fc6e974be28f2e0f5b9b2bfbccdffaa5327a4a0:
net_cls.classid: 0
net_prio.prioidx: 3
net_prio.ifpriomap: lo 0
	enp0s3 0
	docker0 0
	veth18981d6 0

Mounts

Click here for the directory mounted by this Container process

$ cat /proc/$CPID/mounts
overlay / overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/JVWDDYIF5YQFGNGL3PDNWJ4A4M:/var/lib/docker/overlay2/l/TQBVPLJN6SEMDRW7VD5FBARG4F,upperdir=/var/lib/docker/overlay2/ae3eab82db4efee497c2d69a4ad18a8cfc816ce0d61296fee7b6f611d7f6ebb3/diff,workdir=/var/lib/docker/overlay2/ae3eab82db4efee497c2d69a4ad18a8cfc816ce0d61296fee7b6f611d7f6ebb3/work,xino=off 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev tmpfs rw,nosuid,size=65536k,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0
sysfs /sys sysfs ro,nosuid,nodev,noexec,relatime 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,relatime,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup ro,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0
cgroup /sys/fs/cgroup/perf_event cgroup ro,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/cpuset cgroup ro,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup ro,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup ro,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup ro,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/memory cgroup ro,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/pids cgroup ro,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/rdma cgroup ro,nosuid,nodev,noexec,relatime,rdma 0 0
cgroup /sys/fs/cgroup/blkio cgroup ro,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/devices cgroup ro,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup ro,nosuid,nodev,noexec,relatime,freezer 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=65536k 0 0
/dev/sda1 /etc/resolv.conf ext4 rw,relatime 0 0
/dev/sda1 /etc/hostname ext4 rw,relatime 0 0
/dev/sda1 /etc/hosts ext4 rw,relatime 0 0
devpts /dev/console devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0
proc /proc/bus proc ro,relatime 0 0
proc /proc/fs proc ro,relatime 0 0
proc /proc/irq proc ro,relatime 0 0
proc /proc/sys proc ro,relatime 0 0
proc /proc/sysrq-trigger proc ro,relatime 0 0
tmpfs /proc/acpi tmpfs ro,relatime 0 0
tmpfs /proc/kcore tmpfs rw,nosuid,size=65536k,mode=755 0 0
tmpfs /proc/keys tmpfs rw,nosuid,size=65536k,mode=755 0 0
tmpfs /proc/timer_list tmpfs rw,nosuid,size=65536k,mode=755 0 0
tmpfs /proc/sched_debug tmpfs rw,nosuid,size=65536k,mode=755 0 0
tmpfs /proc/scsi tmpfs ro,relatime 0 0
tmpfs /sys/firmware tmpfs ro,relatime 0 0

What we pay attention to here is overlay information.

overlay / overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/JVWDDYIF5YQFGNGL3PDNWJ4A4M:/var/lib/docker/overlay2/l/TQBVPLJN6SEMDRW7VD5FBARG4F,upperdir=/var/lib/docker/overlay2/ae3eab82db4efee497c2d69a4ad18a8cfc816ce0d61296fee7b6f611d7f6ebb3/diff,workdir=/var/lib/docker/overlay2/ae3eab82db4efee497c2d69a4ad18a8cfc816ce0d61296fee7b6f611d7f6ebb3/work,xino=off 0 0

The / var / lib / docker / overlay2 / l / TQBVPLJN6SEMDRW7VD5FBARG4F mounted here is actually the content of the alpine image.

$ cat $(docker image inspect alpine -f '{{ .GraphDriver.Data.UpperDir }}')/../link
TQBVPLJN6SEMDRW7VD5FBARG4F

network

Finally, let's check the network settings. The Container process distinguishes itself into its own network namespace, so use veth to communicate with host mahcine. To do.

First, the network settings on the Container side

$ nsenter -t $CPID -n ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.2  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:ac:11:00:02  txqueuelen 0  (Ethernet)
        RX packets 17  bytes 1366 (1.3 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
$ nsenter -t $CPID -n ip link show type veth
280: eth0@if281: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
$ nsenter -t $CPID -n ip route show dev eth0
default via 172.17.0.1
172.17.0.0/16 proto kernel scope link src 172.17.0.2

host machine side

$ ifconfig docker0
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:ffff:fe14:4bd6  prefixlen 64  scopeid 0x20<link>
        ether 02:42:ff:14:4b:d6  txqueuelen 0  (Ethernet)
        RX packets 8  bytes 433 (433.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 283  bytes 30567 (30.5 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
$ ip addr show type veth
281: veth18981d6@if280: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
    link/ether fa:eb:12:ad:57:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::f8eb:12ff:fead:57d1/64 scope link
       valid_lft forever preferred_lft forever
$ ip route show dev docker0
172.17.0.0/16 proto kernel scope link src 172.17.0.1

The address of Container is 172.17.0.2, and the host is 172.17.0.1 and veth18981d6, which connects the two interfaces. And each route table has the rules of the forwarding route.

Clarify Docker execution order with bpftrace

Now that we've seen the results, how did Docker specifically arrive at those results, for example, when and what function was called by what variable? This is where bpftrace comes into play.

bpftrace is a tool that makes it easy to use Linux Kenel's eBPF functionality with awk-like description. The specific usage is omitted here, but please refer to Doc.

Setting up bpftrace

# bcc
$ apt-get install -y linux-headers-$(uname -r) bison build-essential cmake flex g++ git libelf-dev zlib1g-dev libfl-dev systemtap-sdt-dev binutils-dev llvm-8-dev llvm-8-runtime libclang-8-dev clang-8 arping netperf iperf3 python3-distutils
$ git clone --recurse-submodules https://github.com/iovisor/bcc.git
$ mkdir bcc/build; cd bcc/build
$ cmake -DPYTHON_CMD=python3 ..
$ make -j8 && make install && ldconfig

$ cd ../..

# bpftrace
$ git clone https://github.com/iovisor/bpftrace.git
$ mkdir bpftrace/build; cd bpftrace/build
$ cmake -DHAVE_BCC_PROG_LOAD=ON -DHAVE_BCC_CREATE_MAP=ON -DBUILD_TESTING=OFF ..
$ make -j8 && make install

Trace Docker

You can trace the interaction between docker and the OS with bpftrace. First, create a docker.bt file with this content

docker.bt


#!/usr/bin/env bpftrace

#include <linux/nsproxy.h>
#include <linux/ns_common.h>
#include <linux/utsname.h>
#include <linux/pid_namespace.h>
#include <linux/ipc_namespace.h>
#include <linux/cgroup.h>
#include <net/net_namespace.h>
#include <linux/netdevice.h>

BEGIN { 
    printf("%-12s %-15s %-8s %-10s %s\n", "TIME", "COMMAND", "PID", "ACTION", "CONTENT"); 
}

tracepoint:syscalls:sys_enter_execve { 
    printf("%-12ld %-15s %-8d %-10s ", elapsed , comm, pid, "execve");
    join(args->argv); 
}

tracepoint:syscalls:sys_enter_mount { 
    printf("%-12ld %-15s %-8d %-10s ", elapsed, comm, pid, "mount");
    printf("type=%s, dev=%s, dir=%s\n", str(args->type), str(args->dev_name), str(args->dir_name))
}

kretprobe:create_new_namespaces /comm=="runc:[1:CHILD]"/ { 
    printf("%-12ld %-15s %-8d %-10s ", elapsed, comm, pid, "namespace");
    $nsp = (struct nsproxy *)retval;
    printf("uts=%ld, ipc=%ld, cgroup=%ld, net=%ld, pid=%ld\n", 
        ((struct uts_namespace *)$nsp->uts_ns)->ns.inum,
        ((struct ipc_namespace *)$nsp->ipc_ns)->ns.inum,
        ((struct cgroup_namespace *)$nsp->cgroup_ns)->ns.inum,
        ((struct net *)$nsp->net_ns)->ns.inum,
        ((struct pid_namespace *)$nsp->pid_ns_for_children)->ns.inum);
}

kprobe:veth_newlink {
    printf("%-12ld %-15s %-8d %-10s ", elapsed, comm, pid, "veth");
    printf("name=%s, netns=%ld\n", ((struct net_device *)arg1)->name, ((struct net *)arg0)->ns.inum);
}

And run

$ ./docker.bt
Attaching 5 probes...
TIME         COMMAND         PID      ACTION     CONTENT

Start Container again (another terminal)

$ docker run -ti --rm --name test alpine echo hi

If you return to docker.bt, this content will be output

TIME         COMMAND         PID      ACTION     CONTENT
1827725094   bash            235026   execve     docker run -ti --rm --name test alpine echo hi
1874368200   dockerd         81563    mount      type=overlay, dev=overlay, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
1892118900   dockerd         81563    mount      type=overlay, dev=overlay, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
1904728454   dockerd         81563    mount      type=overlay, dev=overlay, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
1905395722   dockerd         81563    veth       name=veth3d71f6e, netns=4026531992
1912608987   networkd-dispat 235038   execve     /usr/bin/networkctl list --no-pager --no-legend
1913900866   (spawn)         235036   execve     /lib/udev/bridge-network-interface
1917882282   (spawn)         235039   execve     /lib/open-iscsi/net-interface-handler start
1922126799   (spawn)         235040   execve     /lib/udev/bridge-network-interface
1923736757   (spawn)         235041   execve     /lib/systemd/systemd-sysctl --prefix=/net/ipv4/conf/veth3d71f6e --prefix=/net/ipv4/neigh/veth3d71f6e --prefix=/net/ipv6/conf/veth3d71f6e --prefix=/net/ipv6/neigh/veth3d71f6e
1925374081   (spawn)         235042   execve     /lib/open-iscsi/net-interface-handler start
1935733149   containerd      235044   execve     containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
1940074927   containerd-shim 235053   execve     runc --root /var/run/docker/runtime-runc/moby --log /run/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb/log.json --log-format json create --bundle /run/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb --pid-file /run/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb/init.pid --console-socket /tmp/pty046813817/pty.sock 6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb
1940243736   (spawn)         235043   execve     /lib/systemd/systemd-sysctl --prefix=/net/ipv4/conf/vethc30992d --prefix=/net/ipv4/neigh/vethc30992d --prefix=/net/ipv6/conf/vethc30992d --prefix=/net/ipv6/neigh/vethc30992d
1955997514   runc            235060   execve     runc init
1959669403   exe             235060   mount      type=, dev=/proc/self/exe, dir=/var/run/docker/runtime-runc/moby/6975e3344ad32b92a6a09066f3113
1962524451   exe             235060   mount      type=, dev=, dir=/var/run/docker/runtime-runc/moby/6975e3344ad32b92a6a09066f3113
1977110103   runc:[1:CHILD]  235061   namespace  uts=4026532187, ipc=4026532188, cgroup=4026531835, net=4026532191, pid=4026532189
2002233022   runc:[2:INIT]   235062   mount      type=, dev=, dir=/
2002647666   runc:[2:INIT]   235062   mount      type=bind, dev=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2002787106   runc:[2:INIT]   235062   mount      type=proc, dev=proc, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2002848509   runc:[2:INIT]   235062   mount      type=tmpfs, dev=tmpfs, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2003072500   runc:[2:INIT]   235062   mount      type=devpts, dev=devpts, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2006987497   runc:[2:INIT]   235062   mount      type=sysfs, dev=sysfs, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2007808775   runc:[2:INIT]   235062   mount      type=tmpfs, dev=tmpfs, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008001937   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/systemd/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008021032   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/systemd/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008102923   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/perf_event/docker/6975e3344ad32b92a6a09066f31132, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008116742   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/perf_event/docker/6975e3344ad32b92a6a09066f31132, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008189722   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/cpuset/docker/6975e3344ad32b92a6a09066f31132f9b4, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008203301   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/cpuset/docker/6975e3344ad32b92a6a09066f31132f9b4, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008284557   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/cpu,cpuacct/docker/6975e3344ad32b92a6a09066f3113, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008300251   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/cpu,cpuacct/docker/6975e3344ad32b92a6a09066f3113, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008365239   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/net_cls,net_prio/docker/6975e3344ad32b92a6a09066, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008397968   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/net_cls,net_prio/docker/6975e3344ad32b92a6a09066, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008471763   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/hugetlb/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008489627   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/hugetlb/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008575004   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/memory/docker/6975e3344ad32b92a6a09066f31132f9b4, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008588697   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/memory/docker/6975e3344ad32b92a6a09066f31132f9b4, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008655100   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/pids/docker/6975e3344ad32b92a6a09066f31132f9b4e3, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008667811   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/pids/docker/6975e3344ad32b92a6a09066f31132f9b4e3, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008731445   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/rdma, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008744138   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/rdma, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008792554   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/blkio/docker/6975e3344ad32b92a6a09066f31132f9b4e, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008805034   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/blkio/docker/6975e3344ad32b92a6a09066f31132f9b4e, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008855551   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/devices/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008868073   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/devices/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008919459   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/freezer/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008932321   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup/freezer/docker/6975e3344ad32b92a6a09066f31132f9b, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008958905   runc:[2:INIT]   235062   mount      type=bind, dev=/sys/fs/cgroup, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2008977832   runc:[2:INIT]   235062   mount      type=mqueue, dev=mqueue, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009006418   runc:[2:INIT]   235062   mount      type=tmpfs, dev=shm, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009102662   runc:[2:INIT]   235062   mount      type=bind, dev=/var/lib/docker/containers/6975e3344ad32b92a6a09066f31132f9b4e3, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009116671   runc:[2:INIT]   235062   mount      type=, dev=, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009152544   runc:[2:INIT]   235062   mount      type=bind, dev=/var/lib/docker/containers/6975e3344ad32b92a6a09066f31132f9b4e3, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009184117   runc:[2:INIT]   235062   mount      type=, dev=, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009256053   runc:[2:INIT]   235062   mount      type=bind, dev=/var/lib/docker/containers/6975e3344ad32b92a6a09066f31132f9b4e3, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2009268672   runc:[2:INIT]   235062   mount      type=, dev=, dir=/var/lib/docker/overlay2/ec1fbee3aa02f9596794e45afddaddf75028e0
2012566089   runc            235085   execve     libnetwork-setkey -exec-root=/var/run/docker 6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb ee4af79b8952
2055580696   dockerd         81563    mount      type=bind, dev=/proc/235062/ns/net, dir=/var/run/docker/netns/53666134ee2d
2058245033   dockerd         235093   execve     set-ipv6 /var/run/docker/netns/53666134ee2d all false
2117481953   (spawn)         235100   execve     /lib/open-iscsi/net-interface-handler stop
2135647000   runc:[2:INIT]   235062   mount      type=, dev=, dir=.
2135881831   runc:[2:INIT]   235062   mount      type=bind, dev=/dev/pts/0, dir=/dev/console
2135959898   runc:[2:INIT]   235062   mount      type=, dev=/proc/bus, dir=/proc/bus
2135974823   runc:[2:INIT]   235062   mount      type=, dev=/proc/bus, dir=/proc/bus
2135984627   runc:[2:INIT]   235062   mount      type=, dev=/proc/fs, dir=/proc/fs
2135995698   runc:[2:INIT]   235062   mount      type=, dev=/proc/fs, dir=/proc/fs
2136004311   runc:[2:INIT]   235062   mount      type=, dev=/proc/irq, dir=/proc/irq
2136017445   runc:[2:INIT]   235062   mount      type=, dev=/proc/irq, dir=/proc/irq
2136025919   runc:[2:INIT]   235062   mount      type=, dev=/proc/sys, dir=/proc/sys
2136036802   runc:[2:INIT]   235062   mount      type=, dev=/proc/sys, dir=/proc/sys
2136045583   runc:[2:INIT]   235062   mount      type=, dev=/proc/sysrq-trigger, dir=/proc/sysrq-trigger
2136057589   runc:[2:INIT]   235062   mount      type=, dev=/proc/sysrq-trigger, dir=/proc/sysrq-trigger
2136066994   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/asound
2136076692   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/acpi
2136145362   runc:[2:INIT]   235062   mount      type=tmpfs, dev=tmpfs, dir=/proc/acpi
2136198403   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/kcore
2136211095   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/keys
2136271650   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/latency_stats
2136281142   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/timer_list
2136292991   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/timer_stats
2136302265   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/sched_debug
2136312542   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/proc/scsi
2136411156   runc:[2:INIT]   235062   mount      type=tmpfs, dev=tmpfs, dir=/proc/scsi
2136458345   runc:[2:INIT]   235062   mount      type=, dev=/dev/null, dir=/sys/firmware
2136537743   runc:[2:INIT]   235062   mount      type=tmpfs, dev=tmpfs, dir=/sys/firmware
2325058622   containerd-shim 235102   execve     runc --root /var/run/docker/runtime-runc/moby --log /run/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb/log.json --log-format json start 6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb
2334946680   runc:[2:INIT]   235062   execve     echo hi
2357752100   containerd-shim 235108   execve     /usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby
2389193617   containerd-shim 235116   execve     runc --root /var/run/docker/runtime-runc/moby --log /run/containerd/io.containerd.runtime.v1.linux/moby/6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb/log.json --log-format json delete 6975e3344ad32b92a6a09066f31132f9b4e3d5c65e1b827ed91486f1128c17eb
2438983541   (spawn)         235129   execve     /lib/udev/bridge-network-interface
2441125682   networkd-dispat 235130   execve     /usr/bin/networkctl list --no-pager --no-legend
2441917196   (spawn)         235131   execve     /lib/open-iscsi/net-interface-handler start
2445708579   (spawn)         235132   execve     /lib/systemd/systemd-sysctl --prefix=/net/ipv4/conf/vethc30992d --prefix=/net/ipv4/neigh/vethc30992d --prefix=/net/ipv6/conf/vethc30992d --prefix=/net/ipv6/neigh/vethc30992d
2460630127   networkd-dispat 235133   execve     /usr/bin/networkctl list --no-pager --no-legend
2469750858   (spawn)         235134   execve     /lib/open-iscsi/net-interface-handler stop
2470070134   (spawn)         235135   execve     /lib/open-iscsi/net-interface-handler stop

This is self-explanatory what Docker did to run this Container. To summarize briefly, here is the order

Reference material

Recommended Posts

Check out how Docker works with bpftrace
How to install python3 with docker centos
How to delete log with Docker, not to collect log
Tetris with bpftrace
[Python] OpenCV environment construction with Docker (cv2.imshow () also works)
How to share folders with Docker and Windows with tensorflow
Tftp server with Docker
Use python with docker
Proxy server with Docker
Hello, World with Docker
Domain check with Python
Check version with python
How to check ORM behavior in one file with django