I want to try machine learning → Tensorflow, right? I want to try it interactively on the Web → Jupyter, right? I want to build infrastructure easily → Docker, right? So, I tried it, but I got stuck in where to set the password, so I will write the solution
Those who apply to the following
――I'm an infrastructure engineer, but I want to try machine learning --Docker I've been touching it recently, but I don't know the details --Python I'm touching it for a while, but I don't know much --Jupyter Is that okay? I don't know
Basically I would like to try it on my current MacBook Air, but obviously I don't have enough resources. So, if you want to do it firmly, the best solution is on the cloud such as AWS or GCP. I want to hit it locally and then bring it to the cloud, so the easiest and most popular thing is Docker. Building Jupyter is a hassle, so it's really easy to use a container that contains everything. (If you try to put it in Ansible or Chef, it will get stuck in various ways ...)
First, get the image. There are various types of Jupyter images below, so choose one according to your purpose. https://github.com/jupyter/docker-stacks
I want to use tensorflow this time, so I will use the following. https://github.com/jupyter/docker-stacks/tree/master/tensorflow-notebook
docker pull let's pull
$ docker pull jupyter/tensorflow-notebook
Unable to find image 'jupyter/tensorflow-notebook:latest' locally
latest: Pulling from jupyter/tensorflow-notebook
75a822cd7888: Pulling fs layer
f74dd7a42a1f: Pulling fs layer
7b03c47d9699: Pull complete
ecef67eb4d88: Pull complete
bd6c8cee2aa9: Pull complete
8ecf884c467a: Pull complete
7dc68d7c7ffd: Pull complete
3369a456e292: Pull complete
e626c4a34300: Pull complete
619d5c673fb7: Pull complete
eff746881b74: Pull complete
fae5187ea9e1: Pull complete
e1fd76e4adb2: Pull complete
b5d2ba91f6a5: Pull complete
5804e217f19f: Pull complete
7719a427e59f: Pull complete
33ea9d473883: Pull complete
d28a865b24f6: Pull complete
7e2fa13ce090: Pull complete
15b63cba13fd: Pull complete
adcb147a9782: Pull complete
ce56ab01574f: Pull complete
aa6a6ae0b01e: Pull complete
2c54a9d73fcd: Pull complete
47c2872f3870: Pull complete
Digest: sha256:dd7b9e81c167647a87300043f21e3b37eee6d463815b86db9976c299cd4cc046
Status: Downloaded newer image for jupyter/tensorflow-notebook:latest
101e77006b40935607969dbad87e4540719ec55e66168d507f0ea8bb04ffe713
You may get an error as shown below.
$ docker pull jupyter/tensorflow-notebook
Unable to find image 'jupyter/tensorflow-notebook:latest' locally
latest: Pulling from jupyter/tensorflow-notebook
75a822cd7888: Pulling fs layer
f74dd7a42a1f: Pulling fs layer
f74dd7a42a1f: Pull complete
ecef67eb4d88: Download complete
bd6c8cee2aa9: Download complete
8ecf884c467a: Download complete
7dc68d7c7ffd: Download complete
3369a456e292: Download complete
e626c4a34300: Download complete
619d5c673fb7: Download complete
eff746881b74: Download complete
fae5187ea9e1: Download complete
e1fd76e4adb2: Download complete
b5d2ba91f6a5: Downloading 96.62 MB/660.8 MB
5804e217f19f: Download complete
7719a427e59f: Downloading 126.5 MB/281.6 MB
33ea9d473883: Download complete
d28a865b24f6: Downloading 314.1 MB/314.1 MB
7e2fa13ce090: Waiting
15b63cba13fd: Waiting
adcb147a9782: Waiting
ce56ab01574f: Waiting
aa6a6ae0b01e: Waiting
2c54a9d73fcd: Waiting
47c2872f3870: Waiting
docker: write /var/lib/docker/tmp/GetImageBlob222925529: no space left on device.
See 'docker run --help'.
The cause is that the image is about 5G, so it seems that the capacity is insufficient. So, if you delete other images, it's OK (neural style is too big)
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
poppen/trusty-ansible-serverspec latest e84a5c7341fd 10 weeks ago 691 MB
flyinprogrammer/serverspec latest 1b34b62345b3 12 weeks ago 154.4 MB
somatic/torch-fast-neural-style latest 97645d9c69e8 3 months ago 7.323 GB
amazon/amazon-ecs-agent latest a76825ffa321 3 months ago 10.84 MB
elenaalexandrovna/opencv-python3 latest 910fcd0fa477 4 months ago 594.7 MB
trafferty/docker-ipython-opencv latest 686f355ae522 19 months ago 8.676 GB
$ docker rmi a76825ffa321 910fcd0fa477 97645d9c69e8
Untagged: amazon/amazon-ecs-agent:latest
Untagged: amazon/amazon-ecs-agent@sha256:391a45a5b69a8d9fe1844310c4cf90e82e31cb167a93eb94d5a49b2b2456ad46
Deleted: sha256:a76825ffa321686b2df70a0be52fe5ecd52006a347cee5f07635b188655a5e86
...
$ docker run -it --rm -p 8888:8888 jupyter/tensorflow-notebook
1b11efc02512ab60a26909ced181a16e01db57160607bb5c54c2acacae30a20b
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fb31e1a2fda1 jupyter/tensorflow-notebook "tini -- start-notebo" 6 seconds ago Up 1 seconds 0.0.0.0:8888->8888/tcp thirsty_kilby
At this rate, even if you access http: // localhost: 8888, you will only be looking at the password screen.
Launch bash in the container with docker exec
$ docker exec -it fb31e1a2fda1 /bin/bash
Run it from python, type in the password you want to set twice, and get the hash value of the password. In the example below, the hash value is sha1: hogehoge: fugafuga.
< python -c 'from notebook.auth import passwd;print(passwd())'
Enter password:
Verify password:
sha1:hogehoge:fugafuga
jovyan@fb31e1a2fda1:~/work$ exit
exit
If you attach docker, it will attach to the standard input / output of start-notebook.sh that is executed when the container is started, and you cannot type python commands.
$ docker attach fb31e1a2fda1
★ I can't execute any command (just looking at the log)
ç^C[I 13:28:14.809 NotebookApp] Interrupted...
[I 13:28:14.810 NotebookApp] Shutting down kernels
$
Delete this container as it is no longer used
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fb31e1a2fda1 jupyter/tensorflow-notebook "tini -- start-notebo" 10 minutes ago Up 10 minutes 0.0.0.0:32768->8888/tcp tender_snyder
$ docker stop fb31e1a2fda1
fb31e1a2fda1
$ docker rm fb31e1a2fda1
fb31e1a2fda1
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Finally start the container with the password set Put the hash value you got earlier, sha1: hogehoge: fugafuga, into the value of the option NotebookApp.password in start-notebook.sh and execute it.
docker run -d -p 8888:8888 jupyter/tensorflow-notebook start-notebook.sh --NotebookApp.password='sha1:hogehoge:fugafuga'
Now you can go to http: // localhost: 8888 and enter the password you set above to log in. After that, please play with MNIST.
You can use the openssl command, right? I did the following, but it didn't work
$ echo [password] | openssl sha1
I thought sha1 was sha1, but why? Is the format different?