Raspberry Pi Kubernetes and Docker downgraded story

0. Introduction

Using the winter vacation from 2020 to 2021 Kubernetes environment by Raspberry Pi that exists a lot in Qiita I tried to build it with reference to the posts of various seniors. IMG_4092.jpg But. .. .. .. .. I was able to install it, but it's super unstable. It's beyond the control of beginners Eventually it's a story of a loser who will lower the version of Kubernetes and Docker.

1. At the time of installation

1-1. Docker edition

Install immediately!

$ curl -sSL https://get.docker.com/ | sh
# Executing docker install script, commit: 3d8fe77c2c46c5b7571f94b42793905e5b3e42e4

--- Omitted ---

+ sudo -E sh -c docker version
Client: Docker Engine - Community
 Version:           20.10.1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        831ebea
 Built:             Tue Dec 15 04:35:27 2020
 OS/Arch:           linux/arm
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       f001486
  Built:            Tue Dec 15 04:33:02 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
--- The following abbreviations ---

So the version of Docker is 20.10.1 When I think about it later, it was off here. It's not supported by Kubernetes now (end of 2020), but it will be somehow ... Install it on the Raspberry Pi that will become a worker in the same way!

1-2. Kubernetes

Now that Docker has been installed, Kubernetes! After completing Docker's usermod, apt-mark, google's apt-key import, etc.

$ sudo apt-get install kubelet kubeadm kubectl kubernetes-cni

--- Omitted ---

Unpacking kubeadm (1.20.1-00) ...
Setting up conntrack (1:1.4.5-2) ...
Setting up kubectl (1.20.1-00) ...
Setting up ebtables (2.0.10.4+snapshot20181205-3) ...
Setting up socat (1.7.3.2-2) ...
Setting up cri-tools (1.13.0-01) ...
Setting up kubernetes-cni (0.8.7-00) ...
Setting up kubelet (1.20.1-00) ...```
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.
Setting up kubeadm (1.20.1-00) ...
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for libc-bin (2.28-10+rpi1) ...

Well, this is a normal installation. So Kubernetes is 1.20.1。 Install it on the Raspberry Pi that becomes a worker in the same way

The battle by beginners begins from here w

#2.I can't do kubeadm init The Raspberry Pi I used this time Raspberry Pi 4 Model B / 8GB That's why the memory looks like this even with a GUI OS

$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7.7Gi       617Mi       5.8Gi       574Mi       1.3Gi       6.3Gi
Swap:            0B          0B          0B

That's why I immediately started kubeadm init with Raspberry Pi to become a master ~

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.20.1
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.1. Latest validated version: 19.03
	[WARNING SystemVerification]: missing optional cgroups: hugetlb
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR Mem]: the system RAM (1 MB) is less than the minimum 1700 MB
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

What's this. .. .. The version of Docker is high! As good(Not good) This system has only 1MB of memoryHowever, at least 1700MB is required! What do you mean? ?? ?? Where did the free memory go? .. .. That's why I tried various fishing, but I'm not sure, so Optional 　--ignore-preflight-errors=Mem Let's specify, ignore the error and run away w

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=Mem
[init] Using Kubernetes version: v1.20.1
[preflight] Running pre-flight checks
	[WARNING Mem]: the system RAM (1 MB) is less than the minimum 1700 MB
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.1. Latest validated version: 19.03
	[WARNING SystemVerification]: missing optional cgroups: hugetlb
[preflight] Pulling images required for setting up a Kubernetes cluster

 --- Omitted ---

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

 --- The following abbreviations ---

It seems that it was installed successfully this time(I can't do it) After completing various environment settings, kubeadm join from each Raspberry Pi of Worker!!!

Let's take a look at the node as soon as we can join!

$ kubectl get nodes
NAME     STATUS   ROLES                  AGE     VERSION
k8s-m    Ready    control-plane,master   5m23s   v1.20.1
k8s-w1   Ready    <none>                 3m56s   v1.20.1
k8s-w2   Ready    <none>                 3m41s   v1.20.1
k8s-w3   Ready    <none>                 3m20s   v1.20.1

Hmmm, it works normally ~ (And I was thinking at this time ...)

#3.Something is wrong Since all nodes were seen from master, k8s installation, scaryAnd Eat a meal with a smile and come back I wonder if I will try the node again ~

$ kubectl get node
The connection to the server <IP>:6443 was refused - did you specify the right host or port?
$ kubectl get node
The connection to the server <IP>:6443 was refused - did you specify the right host or port?
...

 --- A few minutes in this state ---

$ kubectl get node
Unable to connect to the server: net/http: TLS handshake timeout
$ kubectl get node
Unable to connect to the server: net/http: TLS handshake timeout
...

 --- Several tens of seconds in this state ---

$ kubectl get node
NAME     STATUS   ROLES                  AGE   VERSION
k8s-m    Ready    control-plane,master   1h    v1.20.1
k8s-w1   Ready    worker                 1h    v1.20.1
k8s-w2   Ready    worker                 1h    v1.20.1
k8s-w3   Ready    worker                 1h    v1.20.1

 --- Resurrection ---

?? ?? ?? what? ?? ?? Even if I look at the log, I get a Fatal Error and I can see that it is restarting, but it is uncontrollable. .. .. The above state is repeated every 10 minutes, so it's hard to use this. .. .. As a result of a lot of trouble, I decided to lower the version.

#4. Kubernetes uninstall master,worker Performs on all nodes. I think it's better to stop the service. Musimushi w ###4-1.Delete Kubernetes settings If you don't do this, you will see the old settings after reinstallation.>me

$ sudo kubeadm reset

###4-2.Release hold Because I was trying to prevent the version from going up without permission by apt update etc. First, check the hold status and unhold the corresponding item.

$ sudo apt-mark showhold
$ sudo apt-mark unhold kubelet kubeadm kubectl kubernetes-cni

###4-2.Uninstall Kubernetes Purge and remove related items with autoremove

$ sudo apt-get purge -y kubeadm kubectl kubelet kubernetes-cni kube*
$ sudo apt-get autoremove -y

###4-3.Raspberry Pi restart I feel uncomfortable, so I restarted all Raspberry Pis w (In fact, here is Kubernetes 1.19.I put in 6, but The connection refused problem of the get node above did not improve w)

#5.Uninstall Docker I will do this for all ###5-1.Release hold Since Docker also used apt update etc. to prevent the version from going up without permission First, check the hold status and unhold the corresponding item.

$ sudo apt-mark showhold
$ sudo apt-mark unhold docker-ce docker-ce-cli

###5-2.Uninstall Docker purge and delete related

$ sudo apt-get purge -y docker-ce docker-ce-cli
$ sudo apt-get autoremove -y

###5-3.Raspberry Pi restart I feel uncomfortable, so I restarted all Raspberry Pis w

#6. re-install Docker 19.03 series, Kubernetes 1.Try to reinstall with 19 series ###6-1. Docker #####6-1-1.Version selection First 19.Let's find out what is the newest in the 03 series.

$ apt-cache madison docker-ce
  docker-ce | 5:20.10.1~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
  docker-ce | 5:20.10.0~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
  docker-ce | 5:19.03.14~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
  docker-ce | 5:19.03.13~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
  docker-ce | 5:19.03.12~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
 --- Omitted ---
  docker-ce | 5:19.03.2~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages
  docker-ce | 5:19.03.1~3-0~raspbian-buster | https://download.docker.com/linux/raspbian buster/stable armhf Packages

19.03 series 　5:19.03.14~3-0~raspbian-buster Looks good. (Just in case, docker-ce-I also checked cli, but the same 19.03.Was 14) so,Docker is 19.03.14Decided to!

#####6-1-2.Install by specifying the version

$ sudo apt-get install -y --no-install-recommends docker-ce=5:19.03.14~3-0~raspbian-buster docker-ce-cli=5:19.03.14~3-0~raspbian-buster

#####6-1-3.Verification

$ docker version
Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275d40
 Built:             Tue Dec  1 19:21:06 2020
 OS/Arch:           linux/arm
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:19:00 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

###6-2. Kubernetes #####6-2-1.Version selection Also check the version

$ sudo apt-cache madison kubeadm
    kubeadm |  1.20.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |  1.20.0-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |  1.19.6-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |  1.19.5-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |  1.19.4-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |  1.19.3-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 --- Omitted ---
    kubeadm |   1.6.0-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |   1.5.7-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubeadm |   1.5.6-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
$
$
$ sudo apt-cache madison kubelet
    kubelet |  1.20.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |  1.20.0-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |  1.19.6-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |  1.19.5-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |  1.19.4-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |  1.19.3-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 --- Omitted ---
    kubelet |   1.5.2-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |   1.5.1-01 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubelet |   1.5.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
$
$
$ sudo apt-cache madison kubectl
    kubectl |  1.20.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |  1.20.0-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |  1.19.6-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |  1.19.5-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |  1.19.4-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |  1.19.3-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 --- Omitted ---
    kubectl |   1.5.3-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |   1.5.2-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
    kubectl |   1.5.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages

1.19 series 　1.19.6-00 Looks good. so,Kubernetes is 1.19.6Decided to! kubernetes-cni is. .. ..

$ sudo apt-cache madison kubernetes-cni
 kubernetes-cni |   0.8.7-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 kubernetes-cni |   0.8.6-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 kubernetes-cni |   0.7.5-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 kubernetes-cni |   0.6.0-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 kubernetes-cni |   0.5.1-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages
 kubernetes-cni | 0.3.0.1-07a8a2-00 | http://apt.kubernetes.io kubernetes-xenial/main armhf Packages

The latest is fine w (appropriate)

#####6-2-2.Install by specifying the version

sudo apt-get install -y --no-install-recommends kubelet=1.19.6-00  kubeadm=1.19.6-00 kubectl=1.19.6-00 kubernetes-cni

#####6-2-3.Version confirmation

$ kubelet --version
Kubernetes v1.19.6
$
$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.6", GitCommit:"fbf646b339dc52336b55d8ec85c181981b86331a", GitTreeState:"clean", BuildDate:"2020-12-18T12:07:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/arm"}
$
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.6", GitCommit:"fbf646b339dc52336b55d8ec85c181981b86331a", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:30Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/arm"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.6", GitCommit:"fbf646b339dc52336b55d8ec85c181981b86331a", GitTreeState:"clean", BuildDate:"2020-12-18T12:01:36Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/arm"}

###6-3.so Safe 　Docker:19.03.14 　Kubernetes:1.19.6 It became-(^O^)／

So, kubeadm init also has some WARNING, but Memory 1MB I was able to install without any problems!

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16

I0102 01:10:31.569593    5226 version.go:252] remote version is much newer: v1.20.1; falling back to: stable-1.19
W0102 01:10:31.967024    5226 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.19.6
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: missing optional cgroups: hugetlb
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

get node no longer repeats connection refused-(^O^)／

$ kubectl get node
NAME     STATUS   ROLES    AGE     VERSION
k8s-m    Ready    master   10h11m  v1.19.6
k8s-w1   Ready    <none>   10h15m  v1.19.6
k8s-w2   Ready    <none>   10h16m  v1.19.6
k8s-w3   Ready    <none>   10h18m  v1.19.6

That's it.

Recommended Posts

Raspberry Pi Kubernetes and Docker downgraded story
Install Docker on Raspberry Pi 4 and Raspberry Pi OS 64bit
Install Docker on Raspberry Pi
kubernetes + docker
Install Docker and docker-compose on Raspberry Pi 4, Linux (Debian) and Windows 10, respectively
Put Kanban in Docker of Raspberry Pi 3
I installed Docker on my Raspberry Pi 3
Launched Redmine with Docker on Raspberry Pi 3
Radiko recording server on Raspberry Pi 4 (Docker unused)
I couldn't install docker with raspberry pi2 b +.
[Raspberry Pi] Try to link Apache2 and Tomcat
Send emails using Docker container on Raspberry Pi 3
CI/CD pipeline and Docker
Docker installation and initialization
Docker terms and commands
The key to running Docker on Raspberry Pi 4 (Ubuntu server 20.04)
Put Redmine docker container in Raspberry Pi 4 Model B 2GB
Build a NAS with DLNA function at the speed of a second with Raspberry Pi and Docker Compose