[PYTHON] I tried to build an environment where work in the Docker container on the remote server can be done directly from the local VS Code with SSH connection
This is Ishikawa from NTT DoCoMo. It is my second year as a member of society.
Recently, due to various reasons, the number of companies and schools that recommend working from home or researching at home has increased.
Under such circumstances, the number of people who work from home has increased significantly, and at this time, if I was focusing on building an environment for remote development, I could feel quite good, so please refer to it.
You will be able to work inside the Docker container on the remote server as if you were working locally.
As a premise of this environment construction,
--Local PC is Mac --VSCode on Mac is already installed --Docker is already installed on the remote server
will do.
There is also a method using Docker Desktop, but I found it a little troublesome, so this time I will introduce a method that does not use it.
The completed environment
You will be able to perform the following operations in the Docker container set up on the remote server only on the VS Code screen of the local PC.
--Maintenance of execution environment --Editing the program file --Program execution --Run Jupyter Notebook
In addition to this, you can edit and transfer files in the container using Cyberduck and calculate using GPU.
Creating a container
Then, how to create an environment immediately.
First, let's make an SSH connection on the remote server.
Here, the connection by RSA key authentication is used. (* The setting method of RSA key authentication is general and okay, so it is omitted here.)
After logging in to the remote server with an SSH connection, create a docker file like the one below in any location on the remote server. The file name is "dockerfile".
#Image of the base. The following are suitable for my GPU environment, so please choose the one that is appropriate for you.
#For those who don't use GPU"ubuntu:18.04"Etc. are all right.
FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
#The name of the person who made it.
LABEL maintainer "dcm_ishikawa"
#Import the minimum required items and items required for SSH connection.
RUN apt-get update --fix-missing && \
apt-get install -y apt-utils && \
apt-get install -y software-properties-common vim curl unzip htop openssh-server wget procps
#Settings for SSH connection.
# "password"Please change the place to any password.
#Ubuntu 18 is set in various ways.This is the work to make an SSH connection as the root user in 04.
RUN mkdir /var/run/sshd
RUN echo 'root:password' | chpasswd
RUN echo '\nPermitRootLogin yes' >> /etc/ssh/sshd_config
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
#GPU settings. Not required if you are not using a GPU.
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES utility,compute
#Settings for SSH connection.
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
#Allows you to start an SSH connection when you run.
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
After creating the dockerfile, execute the following command in the same location where you saved the dockerfile.
docker image build . -t [imagename]
docker run --name [containername] -d -p 10000:22 -p 18888:8888 [imagename]
- If you use nvidia-docker, please change
dockertonvidia-docker.
[imagename] is any image name and [containername] is any container name.
For -p 10000: 22, 10000 is an arbitrary port number when passing the port to the remote server.
For -p 18888: 8888, 18888 is the port number when accessing with Jupyter Notebook on the browser. Not required if you don't use Jupyter Notebook or if you use Juoyter Notebook on VS Code.
If you type the following command on the remote server and put it in the container, the container creation is successful.
ssh [email protected] -p 10000
You will be asked for a password when connecting, but the password will be the one described in the dockerfile.
Once connected, create ~ / .ssh / authorized_keys in the container and enter the public key created on your local PC.
Once this is done, use ʻexit` to disconnect the SSH connection to the container.
Also, create ~ / .ssh / authorized_keys in the remote server as well, and enter the public key created on the local PC.
This completes the work on the remote server. Also disconnect the SSH connection to the remote server.
VS Code settings
Then it is the setting of the local PC.
Add the following description about the remote server and SSH to the container on the remote server to ~ / .ssh / config on the local PC. (Create this file if it doesn't exist in the first place)
~/.ssh/config
Host Server01 #Arbitrary connection name
HostName xxx.xxx.xxx.xxx #Remote server address
User username #Username on the remote server
IdentityFile ~/.ssh/id_rsa #The path of the created private key
Host Server01:Container #Arbitrary connection name
HostName 127.0.0.1
User root
IdentityFile ~/.ssh/id_rsa #Same as above
Port 10000 #Port number passed when running the container
ProxyCommand ssh -W %h:%p Server01 #Server01 is the same as the one on the first line
Next, install the extension for SSH connection of VS Code from the following URL.
https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh-explorer
Start VSCode and press the remote connection button on the left.
You should see the connection name of the container specified in ~ / .ssh / config in the list. (Here Server01: Container)
Click the folder mark to the right of the connection name to finally complete the VS Code connection to the container.
On the screen that appears, there is a button that says "Open Folder", and you can select any location in the container by pressing it, so select the location you want to add to the workspace.
Thank you for your hard work. This completes the environment construction. The following is how to use it. Please refer.
Command execution
You can open the command line on the container by clicking the buttons in the order of the images.
Python installation
Use wget to download and install installers such as Anaconda and Miniconda from the container command line.
(Example)
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
rm Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
Launch Jupyter Notebook
First, install the Python extension in the container workspace. ** (* In some cases, it is not installed by default!) ** Installation: https://marketplace.visualstudio.com/items?itemName=ms-python.python
Then install Jupyter from the command line inside the container.
pip install jupyter
Type the command to start Jupyter.
When using on VS Code
Open the command palette with command + shift + P and
Enter > Create New Blank Jupyter Notebook and press ʻEnter`.
With a recent update, this is all you need to start. It's amazing.
When using on a browser
Execute the following command on the command line in the container.
jupyter notebook --port 8888 --ip=0.0.0.0 --allow-root
** Note that there are some things that cannot be done with the jupyter notebook command alone. ** **
I think it is convenient to set it with alias.
The following will be returned to the command line, so copy the xxxxxxxxxxxxxxx part.
http://127.0.0.1:8000/?token=xxxxxxxxxxxxxxx
Open your local browser and go to the address bar
Enter [Remote Server IP Address]: 18888 and press ʻEnter. (18888is the port number specified during docker run.) After this, you will be asked for a password or token, so enter thexxxxxxxxxxxxxxx` you copied earlier and you will be able to use Jupyter Notebook in your browser.
Connect with Cyberduck
You will be able to connect from New Connection with the following settings.
--Select SFTP --Server: Remote server address --Port: The port number specified during run (10000 here) --Username: root --Enter password or select SSH private key
When the connection does not work
If your VS Code or command line SSH connection doesn't work, it's likely that you've rebuilt the container several times and have a conflict with known_hosts.
It may be solved by deleting the relevant description in ~ / .ssh / known_hosts on the local PC, or renaming it to ~ / .ssh / known_hosts.old.
At the end
What did you think. I hope this article will help you in your work from home and research. Let's continue to work together and do our best!
Recommended Posts