[LINUX] Rolling update for Docker Swarm

Introduction

This article describes the rolling updates achieved with Docker Swarm.

Kubernetes allows you to perform rolling updates of your application for no downtime updates.

In the operation during maintenance in the conventional on-premises system, one design pattern was to prepare a sorry server, separate it from the LB one by one, update it, and then install it again after the update is completed.

In the cloud-native era, the benefits of rolling updates are great and will continue to be the de facto standard. In this article, I will explain the operation method to reduce downtime as much as possible in the Docker Swarm environment.

Rolling update

This article assumes that you have built a cluster environment with Docker Swarm and set up the Docker registry locally.

For the cluster environment construction by Docker Swarm, refer to Service mesh learned by Docker Swarm.

Build work

For example, if you build the container without rolling update, the ports will be duplicated, so you need to stop the container and build again. Therefore, service downtime cannot be avoided in a single configuration.

However, rolling updates can be built first, which can save time with application releases.

First, deploy the assets to the server that updates the application.

** The point when doing a rolling update is to change the image version in the Docker-compose file. ** **

In the following Docker-compose file excerpt example, the tag name specified by ** image ** is changed.

--Before update

version: '3'

networks:
  MyApp:
  
services:
  app:
    deploy:
      replicas: 2
    image: 127.0.0.1:5000/stackdemo

--At the time of update

version: '3'

networks:
  MyApp:
  
services:
  app:
    deploy:
      replicas: 2
    image: 127.0.0.1:5000/stackdemo_v2

The result of executing the docker images command before the update is as follows.

REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
127.0.0.1:5000/stackdemo   latest              1f25d780708a        4 minutes ago       878MB
redis                      latest              4cdbec704e47        2 weeks ago         98.2MB
registry                   <none>              708bc6af7e5e        2 months ago        25.8MB
node                       12.2-alpine         f391dabf9dce        11 months ago       77.7MB

Execute the following command to build.

# docker-compose build

WARNING: Some services (app, redis) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
redis uses an image, skipping
Building app
Step 1/10 : FROM node:12.2-alpine
 ---> f391dabf9dce

(Omitted)

Successfully built f03dc959d289
Successfully tagged 127.0.0.1:5000/stackdemo_v2:lates

Run the docker images command and verify that a new image has been created.

REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
127.0.0.1:5000/stackdemo_v2   latest              f03dc959d289        3 minutes ago       879MB
127.0.0.1:5000/stackdemo      latest              1f25d780708a        10 minutes ago      878MB
redis                         latest              4cdbec704e47        2 weeks ago         98.2MB
registry                      <none>              708bc6af7e5e        2 months ago        25.8MB
node                          12.2-alpine         f391dabf9dce        11 months ago       77.7MB

Push to your local Docker registry.

# docker-compose push

WARNING: Some services (app, redis) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Pushing app (127.0.0.1:5000/stackdemo_v2:latest)...
The push refers to repository [127.0.0.1:5000/stackdemo_v2]
6b65a73c07f0: Pushed
647209689cbb: Pushed
a5bda7b4ef65: Pushed
80f6c9bc1a2c: Pushed
76e5dca151d7: Mounted from stackdemo
917da41f96aa: Mounted from stackdemo
7d6e2801765d: Mounted from stackdemo
f1b5933fe4b5: Mounted from stackdemo
latest: digest: sha256:84ee56656dc9a9e87d4058bba2c98ebdd3615629258f13c4ac81ff56def69b11 size: 2002

Deploy by service

To deploy with the docker service update command, execute the following command. The arguments specified by options such as --update-delay are examples.

# docker service update --update-delay 5s --update-parallelism 1 --image 127.0.0.1:5000/stackdemo_v2 stackdemo_app

overall progress: 2 out of 2 tasks 
1/2: running   [==================================================>] 
2/2: running   [==================================================>] 
verify: Service converged 

Deploy by stack

To deploy with the docker stack command, execute the following command. It is the same as the normal stack deployment.

# docker stack deploy --compose-file docker-compose.yml stackdemo

Application downgrade

Move to the directory where the original assets are and deploy with the docker service update command or the docker stack command. The following is an example of the docker stack command.

# docker stack deploy --compose-file docker-compose.yml stackdemo

knowledge

Based on the minimum configuration system, it is desirable that the number of ** replicas ** specified in the Docker-compose.yml file before updating is 2 or more.

You can set an update interval by specifying the --update-delay 60s option of the docker service update command. The effect of down dime can be localized by setting two or more containers to operate for one server.

docker_swarm_update.png

The following is the behavior when rolling update is executed with the docker service update command. This article consists of one Master and one Node. Since the number of ** replicas ** specified in the docker-compose.yml file is 2, two application containers will be created on each host.

After executing the rolling update, the application update will start for each container.

stackdemo_app
overall progress: 1 out of 4 tasks 
1/4: running   [==================================================>] 
2/4:   
3/4:   
4/4:   

The --update-delay 60s option of the docker service update command causes application updates to run every 60 seconds.

stackdemo_app
overall progress: 2 out of 4 tasks 
1/4: running   [==================================================>] 
2/4: running   [==================================================>] 
3/4:   
4/4:   

As an example, the API of this application returns version information as a response. The following is the response to access the DNS name of LB, {"version ":" 1.0 "} and {" version ":" 1.1 "} are returned alternately, so the application is updated sequentially. You can see that it has been broken.

{"version":"1.0"}
{"version":"1.1"}
{"version":"1.0"}
{"version":"1.1"}
{"version":"1.0"}
{"version":"1.1"}
overall progress: 3 out of 4 tasks 
1/4: running   [==================================================>] 
2/4: running   [==================================================>] 
3/4: running   [==================================================>] 
4/4: 

Rolling updates for all containers have been completed.

overall progress: 4 out of 4 tasks 
1/4: running   [==================================================>] 
2/4: running   [==================================================>] 
3/4: running   [==================================================>] 
4/4: running   [==================================================>] 
verify: Service converged 

If you run the docker container ps -a command, you can see that the container where the old application was running is stopped and a container for the new application is being created. As for the movement at the time of rolling update, the container of the old application is stopped first and a new container is created.

CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                      PORTS               NAMES
52db94935164        127.0.0.1:5000/stackdemo_v2:latest   "/bin/sh -c 'yarn st…"   38 minutes ago      Up 38 minutes               3001/tcp            stackdemo_app.4.j5vgh0f7azp88zal6rd7gveny
8ea68352da6c        127.0.0.1:5000/stackdemo_v2:latest   "/bin/sh -c 'yarn st…"   41 minutes ago      Up 41 minutes               3001/tcp            stackdemo_app.2.zc2pzhd2xkbc8obvc4apvg7qh
4733dd99b9f4        127.0.0.1:5000/stackdemo:latest      "/bin/sh -c 'yarn st…"   43 minutes ago      Exited (1) 41 minutes ago                       stackdemo_app.2.s0lq2qx6zkj5v51mdnfrl4ygr
fee3bf216379        127.0.0.1:5000/stackdemo:latest      "/bin/sh -c 'yarn st…"   43 minutes ago      Exited (1) 38 minutes ago                       stackdemo_app.4.uymqss60i5ac4ozxa05xfgaw4
4ef0610778f0        redis:latest                         "docker-entrypoint.s…"   43 minutes ago      Up 43 minutes               6379/tcp            stackdemo_redis.2.ls88b5nb1h5qeo0todl6lc7dn
867d0cadeeea        registry:2                           "/entrypoint.sh /etc…"   About an hour ago   Up About an hour            5000/tcp            registry.1.wspi67ubshjo47af8b55kvz10

If you increase the number of ** replicas ** specified in the Docker-compose.yml file, the log output by Docker will be increased by the container.

As a point of Docker Swarm design, we considered downtime but not storage, so if the Docker log runs out of disk, it will be overwhelming. In that case, it is necessary to devise such as separating the disk area used by Docker.

in conclusion

Docker Swarm is a technology that can be used in a production environment, but it is desirable to consider the log output by Docker and operate it after suppressing risk hedging because the behavior may be unstable.

reference

Recommended Posts

Rolling update for Docker Swarm
Overview of Docker (for beginners)
Tips for running Go with docker
Service mesh learned with Docker Swarm
Docker environment update: add Python package