[PYTHON] Introducing Kaggle's Docker Image on Windows to build an environment

Record the clogged part as a reminder when you try to easily build an environment using Kaggle's Docker Image in your windows notebook.

Basically, I refer to the article of "Teniro Graffiti", so please refer to this first, except for Win10 Pro whose OS is windows. And if you are using Docker toolbox, it will be helpful.

Stumble point ① Docker-compose yaml execution does not work due to volume name error

When I created the docker-compose yaml file as in the above article, I got an error.

version: "3"
services:
  jupyter:
    build: .
    volumes:
      - $PWD:/tmp/working
    working_dir: /tmp/working
    ports:
      - 8888:8888
    command: jupyter notebook --ip=0.0.0.0 --allow-root --no-browser

I get the error "volume name is too short, names should be at least two alphanumeric characters". Result of investigation, the notation of the environment variable of volumes in the yaml file is $ PWD is a variable, but in the environment at hand It doesn't work, and it seems better to run the $ {PWD} and the PWD (get the current directory path) command. So it worked better with $ {PWD} / tmp / working instead of $ PWD: / tmp / working (because I'm a Docker beginner, I'd appreciate it if you could comment on the reason for more details).

Stumble point ② I can't connect to Docker with a browser

When I ran it with the yaml file modified by docker-compose up --build, I was able to successfully build the container with the jupyter notebook running in the kaggle analysis environment. However, in the browser "http: // localhost: 8888 /? token = XXXX" Attempting to connect to the jupyter notebook via Docker's 8888 port by entering the URL does not work. In the case of Docker toolbox, you can not connect directly to localhost (local host), get the IP address of the virtual machine running with Docker Toolbox and enter it in the URL instead of localhost [Ingenuity](https://qiita.com / jusotech10 / items / b292ac38197926fc6afa) is required.

> docker-machine ip default
192.168.XX.XX

Then I was able to connect to the jupyter notebook successfully.

Stumble point ③ I can't mount the host OS to the directory properly

The volume part of the yaml file mentioned above represents the directory to mount, but in my environment, the mount did not work and the directory on the host OS (local) and the VM did not synchronize. As a result of the investigation, it was found that the cause was as follows.

Oracle VirtualBox folder sharing is not set
The version of the VM boot file (boot2docker.iso) on the docker side does not match the version of VirtualBox that was installed when docker toolbox was installed.
Launched "Oracle VM VirtualBox Manager" and of the docker machine VM you are currently using You can mount it by checking Details> Shared Folder> Add New Shared Folder> Auto Mount / Persist. Since the docker toolbox for windows goes through VirtualBox, the docker command will not be effective unless you make such settings. ** VM VirtualBox icon ** ** VM VirtualBox Manager ** ** VM details shared folder settings window ** However, even if I set about 1., the mount did not work. The cause is that the version of VirtualBox installed first and the version of the boot file of Docker's VM did not match. (I don't know why this happened) I could change the boot file to match the current version of VirtualBox, but I reinstalled VirtualBox to the latest version and updated the boot file at the same time. The boot file is stored as C: //Users/{user}/.docker/machine/cache/boot2docker.iso. This file is available at ** here **. You can find the version of VirtualBox in Help> About VirtualBox, and you can download the installer at ** here **. Replace the boot file, reinstall VirtualBox, and remove the old docker VM machine with VirtualBox Manager. When you start Docker Quick Start Terminal, it will automatically launch a new docker VM based on the boot file. Although it is a small detail, the difference between the boot file version and the VirtualBox version remains the same, and the version of the configuration file called GuestAdditions.iso used to mount the host OS is different, which causes the mount to fail.

Stumble point ④ I don't want to launch jupyter notebook every time

I created a Docker container with a yaml file based on the Docker image, but since the command to start the jupyter notebook is included every time the container is started, the jupyter notebook will start up with the container each time. At that time, by launching the container with the following command and attaching it, you can access the container with the same settings except that you do not launch jupyter notebook in the yaml file settings described above.

docker run -t -i --name {Container name} -p 8888:8888 -v $PWD:/tmp/working:/tmp/working -w /tmp/working {Docker image name} /bin/bash

After creating various installs and directories, jupyter notebook --ip = 0.0.0.0 --allow-root --no-browser You can also launch and use jupyter notebook with . Similarly, you can create and execute the yaml file that matches the docker command above. (I will omit it here) For reference, the docker commands that are often used are listed below.

#Launch the VM
docker-machine start {VM name}
#List of currently created images
docker images
#List of currently created containers
docker ps -a
#Start the container confirmed above by name or ID
docker start {Container name/Container ID}
#Attach to the started container
docker attach {Container name/Container ID}
#Exit the VM
docker-machine stop {VM name}
#Delete unnecessary images
docker rmi {Image name/Image ID}
#Delete unnecessary containers
docker rm {Container name/Container ID}

Finally

If you want to easily analyze with a commercially available windows notebook, but don't want to pollute the environment, Docker is a very useful software. I have summarized the points that I stumbled upon when I introduced it in general Windows 10 Home Edition. I'm glad if you can use it as a reference.