[PYTHON] (Note) Notes on building TensorFlow environment with Docker Compose (only one container)

Introduction

For my memorandum: sweat: It is a memo when creating an environment of Python + TensorFlow + Keras with Docker Compose (only one container). Please note that this is an article I made for myself, so it may be difficult to understand, information, and technology may be out of date: bow_tone1:

Reference material

I used it as a reference: bow_tone1:

Environment * I think that it will work even if it is not the following Ver, but please note that it is old: no_good_tone2: </ sup>

Ubuntu version

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Docker version

$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:25:46 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:24:19 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Docker-Compose version

$ docker-compose version
docker-compose version 1.25.5, build unknown
docker-py version: 4.2.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.1c  28 May 2019

* For some reason build unknown. I gave up because it seemed to take time: sob: </ sup>

Directory structure

I make it properly for testing. .. ..

dk_ai_dir
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
├── src1

Contents of required files

docker-compose.yml


version: '3'
services:
  dk_ai:
    restart: always
    build: .
    container_name: 'dk_ai'
    tty: true
    volumes:
      #Directory to mount
      - .:/dk_ai_dir
    ports:
      #Host side port: Container side port
      - 7000:7000

Dockerfile


FROM ubuntu:18.04

WORKDIR /dk_ai_dir
COPY requirements.txt /dk_ai_dir

RUN apt-get -y update \
    && apt-get -y upgrade \
    && apt-get install -y --no-install-recommends locales curl python3-distutils vim ca-certificates \
    && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py \
    && python3 get-pip.py \
    && pip install -U pip \
    && localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* \
    && pip install -r requirements.txt --no-cache-dir

ENV LANG en_US.utf8

requirements.txt


Flask==1.1.0
gunicorn==19.9.0
Keras>=2.2.5
numpy==1.16.4
pandas==0.24.2
pillow>=6.2.0
python-dateutil==2.8.0
pytz==2019.1
PyYAML==5.1.1
requests==2.22.0
scikit-learn==0.21.2
sklearn==0.0
matplotlib==3.1.1
tensorboard>=1.14.0
tensorflow>=1.14.0
mecab-python3==0.996.2

docker-compose build (launched in background)

$ docker-compose up -d --build

Display docker-compose image information

$ docker-compose images
Container     Repository       Tag       Image Id       Size  
--------------------------------------------------------------
dk_ai       dk_ai_dir_dk_ai   latest   6f8b190eba7e   2.103 GB

Is it bad to make? It seems that the capacity is quite large: sweat: </ sup>

List of containers

$ docker-compose ps
Name     Command    State           Ports         
--------------------------------------------------
dk_ai   /bin/bash   Up      0.0.0.0:7000->7000/tcp

Connect to container

$ docker-compose exec dk_ai /bin/bash
root@ddc90c65586d:/dk_ai_dir# pwd
/dk_ai_dir

Check the contents to see if it is done

I omitted some because the display of the output result is long: sweat: </ sup>

root@ddc90c65586d:/dk_ai_dir# pip3 list
Package                Version
---------------------- -----------
absl-py                0.9.0
Flask                  1.1.0
gunicorn               19.9.0
h5py                   2.10.0
joblib                 0.14.1
Keras                  2.3.1
Keras-Applications     1.0.8
Keras-Preprocessing    1.1.2
matplotlib             3.1.1
mecab-python3          0.996.2
numpy                  1.16.4
pandas                 0.24.2
Pillow                 7.1.2
pip                    20.1
pyparsing              2.4.7
python-dateutil        2.8.0
requests               2.22.0
rsa                    4.0
scikit-learn           0.21.2
scipy                  1.4.1
setuptools             46.3.0
sklearn                0.0
tensorboard            2.2.1
tensorboard-plugin-wit 1.6.0.post3
tensorflow             2.2.0
tensorflow-estimator   2.2.0
urllib3                1.25.9
wheel                  0.34.2

For the time being, check with the docker command

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
dk_ai_dir_dk_ai     latest              6f8b190eba7e        29 minutes ago      2.1GB
webque-api-img      latest              d0aa942613bc        9 days ago          1.9GB
python              3                   4f7cd4269fa9        2 weeks ago         934MB
ubuntu              18.04               c3c304cb4f22        2 weeks ago         64.2MB
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
ddc90c65586d        dk_ai_dir_dk_ai     "/bin/bash"         30 minutes ago      Up 30 minutes       0.0.0.0:7000->7000/tcp   dk_ai
860f6e3c6376        webque-api-img      "/bin/bash"         8 days ago          Up 2 days           0.0.0.0:8888->8888/tcp   webqueapi

Things other than the one I made this time (dk_ai_dir_dk_ai) are the ones I made in the past. It seems that the capacity is quite large: sweat: </ sup>

Service outage

$ docker-compose stop
Stopping dk_ai ... done
$ docker-compose ps
Name     Command    State    Ports
----------------------------------
dk_ai   /bin/bash   Exit 0        

Start of the service

$ docker-compose start
Starting dk_ai ... done
$ docker-compose ps
Name     Command    State           Ports         
--------------------------------------------------
dk_ai   /bin/bash   Up      0.0.0.0:7000->7000/tcp

When you want to clean the environment

* Reference material is as it is. See reference materials for details: bow_tone1: </ sup>

#Stop & delete
#Container network
docker-compose down

#Container network image
docker-compose down --rmi all

#Container network volume
docker-compose down -v

Recommended Posts

(Note) Notes on building TensorFlow environment with Docker Compose (only one container)
(Note) Notes on building TensorFlow + Flask + Nginx environment with Docker Compose
Note on building your own Miniconda environment
Note: Prepare the environment of CmdStanPy with docker
Building a TensorFlow environment that uses GPU on Windows 10
Launch Django on a Docker container with docker-compose up
Notes on creating a virtual environment with Anaconda Navigator
Let's get started with Python ~ Building an environment on Windows 10 ~
[Django] Build a Django container (Docker) development environment quickly with PyCharm
Prepare python3 environment with Docker
[Tensorflow] Tensorflow environment construction on Windows 10
Try building JupyterHub with Docker
Rails application building with Docker
Building an HPC learning environment using Docker Compose (C, Python, Fortran)
Create a Docker container image with JRE8 / JDK8 on Amazon Linux
Postgres environment construction with Docker I struggled a little, so note
Building a development environment with Maven on Google App Engine [Java]