Python development on Ubuntu on AWS EC2 (using JupyterLab)

Overview

I had the opportunity to use EC2 in my research, so I will write down what I did at that time. I hope it will help people who develop Python on EC2 from now on.

The execution environment is Mac Catalina. (Addition: Even on Windows, the environment could be built by doing the same with Git Bash.)

Step.1 Creating an instance

I used ubuntu 18.04 LST which can be used for free tier. There is an image provided by nvidia, but I tried it there and it didn't work, so I made it from scratch.

The instance type can be a free tier. You don't have to pay a high fee while building the environment. If you want to calculate the heavy load later, change it at that time.

I haven't made any other settings, so it's OK to proceed with "Confirmation and Creation".

Please check the creation of pem file separately. I'm sorry for being unfriendly. After that, we will proceed on the assumption that the pem file has been created.

Step.2 Connect to your instance

If the created instance is running and the status check passes the 2/2 check, try connecting to the instance. Then, EC2 Instance Connect / Session Manager / SSH Client I think you will jump to the page with the tab. Open the tab for that SSH client.

Also, in the terminal, move to the directory where the pem file is located. And in the SSH client tab, After executing the command starting with chmod, Execute a command like ssi -i "~~~ .pem" ubuntu @. You should be able to connect to the created instance.

↑ This is the screen

Step.3 Install Jupyter Lab

Execute the following code on the created instance.

sudo apt update
sudo apt upgrade
sudo apt install -y software-properties-common
sudo apt-add-repository -y ppa:git-core/ppa
sudo apt install -y git
sudo apt install -y python3-pip
sudo pip3 install --upgrade pip
sudo pip3 install jupyterlab
sudo pip3 install ipywidgets
jupyter nbextension enable --py widgetsnbextension

You have now installed jupyterLab on the instance you just created.

Step.4 Connect to Jupyter Lab

If you can do this, stop the instance once, and if you can confirm the stop, start it again.

If the check is passed as in Step.1, connect with the SSH client as before, but add "-L 8889: 127.0.0.1:8889" to the command.

ssh -i "***.pem" -L 8889:127.0.0.1:8889 ubuntu@ec2-***-***-***-***.*********.compute.amazonaws.com

After connecting, execute the following command on the instance to start jupyterLab.

jupyter-lab --ip=0.0.0.0 --port=8889 --no-browser --allow-root --NotebookApp.token=''

If you don't get any errors, in your browser

127.0.0.1:8889

Try connecting to. I think that you can connect to the jupyterLab running on the launched instance.

Finally

I hope there are no errors. After that, I installed packages such as tensorflow, but there are various problems ... Currently, that problem has been solved, so I think it will be helpful if you ask at ↓. (In some cases, I will write it as a separate article) Thank you for reading.

Referenced articles etc. (Thank you !!)

Operate the jupyter notebook on the SSH destination server on the local PC https://sishida21.github.io/2019/12/12/remote-jupyter-notebook/

[Explanation with image] Create Anaconda environment with Docker and use VS Code in the container https://qiita.com/komiya_____/items/96c14485eb035701e218

I built the environment by referring to the dockerfile I created a long time ago, so there must be other articles that I referred to at that time ... but I can't remember ... I'm sorry.