[LINUX] Carry a Docker container

Overview

Using a USB memory, we will build an environment where you can run the Docker container anywhere you have a Windows PC without any administrator privileges.

About USB memory

If you do not select a fast USB3.0 compatible one, it will be pulled by the writing speed and the operation will slow down.

I think that it is painful if the value of random writing (lower right two) of CrystalDiskMark does not exceed 1.0 (* There are individual differences in experience)

By the way, I use U3-MAX2 / 32K of I-O DATA.

If you can make it a little bulky, you can use a portable SSD and there should be no problem in terms of speed unless it is a defective product.

In the case of USB memory up to 32GB, it is generally formatted in FAT32 in the initial state, but FAT32 has a limit of up to 4G per file, and the virtual storage composed of 1 file will soon run out and a write error will occur. Therefore, it is necessary to reformat with NTFS or exFAT.

Also, due to its nature, it writes a lot, so be prepared to break it at any time.

Installation of virtualization software

Qemu

Docker only works on (64bit) Linux, so you need to run Linux with virtualization software, use Qemu for virtualization software.

Qemu's emulation speed is not good, but it can be started without enabling various accelerators that require administrator privileges. In other words, if you don't mind being slow, you can start it without administrator privileges.

WSL2 and other virtualization software require administrator privileges and cannot be operated in a portable manner, and Portable-VirtualBox, which is intended for portable operations, requires administrator privileges.

It is difficult to run a desktop environment such as Gnome without an accelerator, but it works as it is in a CUI (command line) environment. (* There are individual differences in experience) In the Docker area, the UI is basically the responsibility of the Web browser, so it doesn't really matter.

The point that you can emulate a 64-bit PC even on a 32-bit PC / OS is also high.

Download and install

From Official Site Windows version of Qemu (32bit version Download .weilnetz.de / w32 /) or 64bit version).

It seems that the Windows version of Qemu is distributed only in the installer format that requires administrator privileges, so Universal Extractor 2 ([direct link] that supports decompression of the installer ](Https://github.com/Bioruebe/UniExtract2/releases/download/v2.0.0-rc.2/UniExtractRC2.zip) to unzip to a USB memory.

By the way, you can also use it by installing with administrator privileges and copying the installed file to a USB memory.

Linux installation

Alpine Linux

After installing (copying) Qemu on the USB memory, install Linux on the virtual PC.

The Linux distribution to be installed can be anything that supports 64bit and can install Docker, but

I personally recommend Alpine Linux. Installation is explosive. If you try to install a commonly used Linux distro such as Debian from scratch, you have to be prepared for a few hours even with the minimum configuration on Qemu, which is already slow, but the installation will be completed in a few minutes.

Alpine Linux is a distribution that is not afraid of misunderstanding and is focused on weight reduction at the expense of compatibility, and because of its small size, it is often adopted as the base of the official image of Docker, and it is often seen in the Docker area. ..

There are some compatibility difficulties such as not being able to use the software used in other Linux distributions, but if you operate Alpine Linux itself as a Docker host and basically run the software as a Docker container, it will be a problem. It will not be.

iso download

There is an iso download page on the official site, so download the x86_64 version.

Standard (124M) is fine, but we recommend the smaller Virtual (40M) for virtual PCs.

When downloaded, it will have a long file name such as alpine-virt-3.12.0-x86_64.iso, so changing it to alpine.iso will make the rest easier.

Installation

Unlike other virtualization software, Qemu has a mechanism to set and start everything from the command line (although there are various front ends for running with GUI).

Creating a virtual storage file

> qemu-img create -f qcow2 linux.qcow2 20G

The last 20G will be the maximum capacity of virtual storage files, so change it according to your environment. Instead of suddenly securing the full capacity of 20G, the size is increased as needed until the maximum capacity is reached.

Boot with the iso file as the boot disk

See the documentation for Qemu options, I set it as follows:

> qemu-system-x86_64.exe -display sdl -rtc clock=vm,base=utc -smp 2 -boot d -m 2048 -net nic,model=virtio -hda linux.qcow2 -cdrom alpine.iso

If all goes well, a window should pop up and start booting Linux. Booting can take a minute to a few minutes.

Installation

When the boot is finished, the console login screen will appear, but you can log in without a password by typing root and pressing enter.

localhost login: root

After logging in,

# setup-alpine

Start the installation with. You will be asked some questions on the command line, and as you answer, the installation will begin.

Please refer to here for the specific contents. Since it is long, I will omit it.

After installation

# poweroff

Shut down with.

Linux boot / settings

Start-up

> qemu-system-x86_64.exe -display sdl -rtc clock=vm,base=utc -smp 2 -m 2048 -net nic,model=virtio -net user,hostfwd=tcp::22-:22,hostfwd=tcp::2375-:2375,hostfwd=tcp::9000-:9000 -hda linux.qcow2

I'm running this way (except for some).

Connect to the server on Linux in Qemu through the port specified in the part "-net user, hostfwd = tcp :: 22-: 22, hostfwd = tcp :: 2375-: 2375, hostfwd = tcp :: 9000-: 9000" You will be able to do it. It seems to be called port forwarding. (* This is a local connection of the same PC, and the connection from an external PC cannot be done as it is due to the Windows firewall, router, etc., and administrator privileges are required.)

For example, in this case localhost: 22 will allow you to connect to the server listening on port 22 on Linux in Qemu instead of the Windows host.

Here, 22 for SSH, 2375 for Docker CLI, and 9000 for Portainer are assigned. As mentioned above, you can open multiple ports by connecting with, but please note that if you specify a port that is already waiting on another server, Qemu itself will not start with an error.

By the way, it is also effective for connecting the server on the Docker container. In other words, if you enable listening on port 9000 when you start Qemu, you can connect to a Docker container started on a Linux host with port 9000 on listening from a Windows host on port 9000 (confusing ...).

It takes about 1 to 2 minutes to boot Linux, depending on the environment. I'm not sure if it's slow or something like this, but it may be a bottleneck because it will be started and stopped repeatedly in portable operation.

SSH settings

When the login screen appears in the same way as during installation, log in with the root ID and password set during installation.

Since it is inconvenient to copy and paste commands in the Qemu window, SSH can be used.

Alpine Linux comes with an SSH server from the beginning (if you did not answer none in the SSH question during installation), but the SSH server specifications prevent you from logging in as the root user as it is, so here Quickly add the configuration file so that you can log in as root.

# echo "PermitRootLogin yes" >> /etc/ssh/sshd_config

If you want to operate without a password, enter the following as well.

# echo "PermitEmptyPasswords yes" >> /etc/ssh/sshd_config

Shut down and restart Qemu after changing the config file or restart the SSH server

# service sshd restart

Connect

Recent Windows 10 has an Opeh SSH client as standard, but if you use it as it is, it will create a .ssh folder in the user folder and it will not be portable, so do not read the configuration file as shown below and do not check the key , Start with the option not to leave the KnownHosts file.

> ssh -F /dev/null -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@localhost

Telnet connection to Qemu

If you don't want to operate in an unprotected state like the SSH server settings above, you can use Telnet to display the console instead of opening a window with "-display sdl".

> qemu-system-x86_64.exe -display none -serial telnet:localhost:4321,server,nowait -rtc clock=vm,base=utc -smp 2 -m 2048 -net nic,model=virtio -net user,hostfwd=tcp::22-:22,hostfwd=tcp::2375-:2375,hostfwd=tcp::9000-:9000 -hda linux.qcow2

Now, after starting Qemu, telnet to port 4321 and the console will be displayed.

Since the Telnet client does not start by default in Windows 10, Tera Term or Putty Telnet compatible terminal software such as apps / internet / putty_portable (both have portable versions) is required.

Both are very convenient because you can copy and paste commands.

If you always enable this, you can connect if you know the port and it is not much different from the above setting on the SSH server, so enable it only when necessary.

It is safe to use it as a connection until the proper SSH server settings are completed.

If you think about it carefully, TeraTerm has a macro function, and it seems that you can automatically operate Linux with macros as long as you can connect to Telnet, so if you combine it with a batch file, you can automate all the work of this article ... → Done

Docker installation

Alpine Linux has its own package manager called apk, but docker cannot be installed in the initial state, so edit / etc / apk / repositories with vi etc. so that it can be installed.

In the initial state, the community repository on the second line is commented out with #, but delete # to enable it.

/etc/apk/repositories


#/media/cdrom/apks
http://dl-cdn.alpinelinux.org/alpine/v3.12/main
http://dl-cdn.alpinelinux.org/alpine/v3.12/community
#http://dl-cdn.alpinelinux.org/alpine/edge/main
#http://dl-cdn.alpinelinux.org/alpine/edge/community
#http://dl-cdn.alpinelinux.org/alpine/edge/testing

After saving, install docker and docker-compose package with apk

# apk update
# apk add docker docker-compose

After the installation is complete, start docker.

# service docker start

Set it to start automatically at startup.

# rc-update add docker boot

If you want to see if it works, (maybe) download the hello-world image provided for it and run it as a container.

# docker run --rm hello-world

If all goes well, you should see something like the following.

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:49a1c8800c94df04e9658809b006fd8a686cab8028d33cfba2cc049724254202
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Container operation with Docker CLI for Windows

You can also use the Windows version of Docker CLI to control Docker in Qemu from a Windows host.

I can't say anything because I haven't used it, but in the end, I think it will be like Docker for Windows (Docker Tookbox or Docker for Windows) that operates Docker in Linux on virtualization software.

Docker CLI seems to be Basically build by yourself, but [Builded binary](https://github.com/StefanScherer/docker-cli- There are people who have published builder / releases), so if you have trouble building, please.

Docker Compose has Official pre-built binaries, so use that. Please rename it to docker-compose.exe later.

As expected, you can not connect without setting anything, so create the following /etc/docker/daemon.json file so that you can connect from the Windows host. Qemu also needs to change its boot options so that it can connect to port 2375.

/etc/docker/daemon.json


{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}

After creating the file, restart Docker

# service docker restart

On Windows, put the path to the exe you downloaded earlier and set the environment variable to connect to localhost: 2375.

> set DOCKER_HOST=localhost:2375

Try it out to see if it works with docker ps. If it goes well, it will look like this.

> docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                          NAMES
5b07a88001c9        portainer/portainer   "/portainer"             5 days ago          Up 9 minutes        0.0.0.0:9000->9000/tcp                         portainer

To be honest, it doesn't make much sense to just operate Docker from the Windows command line. It is better to operate from Linux directly via SSH or Telnet.

If you are forced to look for merits, is it possible to operate Docker and remote development to a container from Visual Studio Code running on Windows through Docker extension and Remote-Containers extension? There is a way to set up an SSH server and develop remotely with SSH, so that is also a delicate place.

Installation of Portainer

If you find it difficult to manage all Docker containers from the command line, we recommend Portainer, which allows you to manage Docker containers on a web browser.

If you have installed Docker, the installation is completed with one line of the following command.

# docker run -d -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

After the installation is complete, access http: // localhost: 9000 with a web browser and you should see the Portainer login screen.

Please refer to here for the initial settings.

Security

In this case, root operation does not matter much because it works only while the USB memory is connected to the PC and Qemu is operating, and usually only local connection is possible, but basically it is an operation that should not be done. Keep in mind that there is. Also, be careful not to lose or steal your USB memory.

at the end

This completes the environment for carrying a Docker container with a USB memory. Recommended for those who do not have a PC that they can manage freely.

Recommended Posts

Carry a Docker container
Run Matplotlib on a Docker container
How to delete a Docker container
Run matplotlib on a Windows Docker container
Use the Kaggle API inside a Docker container
Try Python interactive mode inside a Docker container
Launch Django on a Docker container with docker-compose up
Build a Docker container and save png from altair
[Django] Build a Django container (Docker) development environment quickly with PyCharm
Generate a Docker image using Fabric
Create a DI Container in Python
Creating a Flask server with Docker
Build a deb file with Docker
Deploy a Django application with Docker
Start Docker container when running Pytest
Create a Docker container image with JRE8 / JDK8 on Amazon Linux
I made a Docker container to use JUMAN ++, KNP, python (for pyKNP).
Run a Python file inside a Docker container on a remote Raspbian via PyCharm
A memo when setting up a Docker container for using JUMAN ++, KNP, python
How to make a container name a subdomain and make it accessible in Docker
Set up a Samba server with Docker
Get a local DynamoDB environment with Docker
Let's scrape a dynamic site with Docker
Create a Python-GUI app in Docker (PySimpleGUI)
Create a web service with Docker + Flask
Use WebDAV in a Portable Docker environment
[Linux] Build a Docker environment with Amazon Linux 2
How to run a Django application on a Docker container (development and production environment)