This article is a sister article of the previously written Python development flow using pipenv, Git and Docker. Now that Pipenv development isn't very active, I think there will be occasions when Poetry is used in projects that use Python. Therefore, I would like to reorganize the development flow using Poetry.
In most cases, you can install it from the package manager that came with your OS.
$ brew install poetry
Arch Linux
$ sudo pacman -S python-poetry
Ubuntu 18.04 Unfortunately, it is not registered in the standard repository, so install it from the official installer.
$ sudo apt install python3 python3-pip
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3
When the installation is complete, put your PATH in ~ / .poetry / bin. Depending on your shell, add the following settings to ~ / .zshrc and ~ / .bashrc.
~/.bashrc
export PATH=${HOME}/.poetry/bin:${PATH}
I'll be using Git and Docker / Docker Compose in the future, so be prepared.
Now it's time to initialize your Python project with poetry.
First, create an appropriate directory and put it in it.
$ mkdir sample-app
$ cd sample-app
You can then initialize the project by running poetry init. It's interactive, so let's answer each one. You will be asked if you want to install the dependent packages on the way, but this time I want to install them individually, so I will skip it.
$ poetry init
This command will guide you through creating your pyproject.toml config.
Package name [sample_app]:
Version [0.1.0]:
Description []: Sample App
Author [Aruneko <[email protected]>, n to skip]:
License []: MIT
Compatible Python versions [^3.7]:
Would you like to define your main dependencies interactively? (yes/no) [yes] no
Would you like to define your development dependencies interactively? (yes/no) [yes] no
Generated file
[tool.poetry]
name = "sample_app"
version = "0.1.0"
description = "Sample App"
authors = ["Aruneko <[email protected]>"]
license = "MIT"
[tool.poetry.dependencies]
python = "^3.7"
[tool.poetry.dev-dependencies]
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
Do you confirm generation? (yes/no) [yes]
When you exit the interactive environment, you will have pyproject.toml. This completes the project initialization.
$ ls
pyproject.toml
Normally, a Python virtual environment created under ~ / Library / Caches / pypoetry / virtualenvs or ~ / .config / poetry will have the .venv directory directly under the project root if the following settings are added. Will be created and will manage the Python environment there. Please set as you like.
$ poetry config virtualenvs.in-project true --local
Please note that this setting is listed in poetry.toml instead of pyproject.toml.
Under Poetry environment, add packages using ʻadd` like npm. Let's develop an application that uses FastAPI as a sample.
$ poetry add fastapi uvicorn
When you install a package with Poetry, the dependent packages that can be resolved at that time and the version information of each package are written to poetry.lock.
Packages that are only used during development are also installed with the -D option, just like npm. For example, let's install the code formatter black.
$ poetry add -D black
By executing the commands so far, you can see that the packages installed so far have been added to the tool.poetry.dependencies and tool.poetry.dev-dependencies sections of pyproject.toml. ..
pyproject.toml
[tool.poetry.dependencies]
python = "^3.7"
fastapi = "^0.52.0"
uvicorn = "^0.11.3"
[tool.poetry.dev-dependencies]
black = "^19.10b0"
From now on, please install the package as needed each time.
Next, initialize Git to manage this project with Git. Let's use gitignore.io to create a .gitignore file first. This time, we will ask you to specify Python and virtualenv. If you need additional settings for .gitignore, edit it here.
Once you have .gitignore, initialize your Git repository, add roughly all the files and commit.
$ curl -o .gitignore https://www.gitignore.io/api/python,virtualenv
$ git init
$ git add pyproject.toml poetry.lock poetry.toml .gitignore
$ git commit -m "Initial Commit"
Use the shell subcommand to enter the Python virtual environment created by Poetry. If you forget to enter the virtual environment, various problems such as not being able to see the package you should have installed will occur, so be sure to check it carefully. To exit, use the ʻexit` command.
$ poetry shell
(.venv)$ exit
Now that we have created the Poetry project, we will finally work on Docker.
Files that do not need to be transferred should be entered in .dockerignore. Be sure to include it, especially if you have set the .venv file to be in the project root. In addition, it is a good idea not to transfer the __pycache__ directory.
.dockerignore
.venv/
__pycache__/
Finally, I will write a Dockerfile. This time, I will introduce the method using Multi Stage Build. It is divided into the procedure of creating requirements.txt in the first half and using it to create a Docker Image with a Python application in the second half.
The reason why you need to do that in the first place is that you need to install Poetry to read poetry.lock, but you don't need to put Poetry in the container where your application is located, so you want to separate them. Because there is an intention. Poetry has the ability to generate requirements.txt, so we'll take advantage of it.
First of all, from the explanation of the first half. To make good use of Docker's caching strategy, I install Poetry first. This will save you the trouble of installing Poetry on every build. Then it just copies pyproject.toml and poetry.lock to generate requirements.txt.
Next is the explanation of the latter half. Copy the resulting requirements.txt from the first half and then use the pip command to install all those packages. If these files are unchanged, the cache will be used to automatically build the next layer, so you can prevent the behavior of downloading dependent packages every time you build. .. This is a recommended configuration because it can save overwhelming build time. After the installation is complete, transfer various Python scripts and write the commands you want to run, and you're done.
FROM python:3.8-slim as builder
WORKDIR /usr/src/app
RUN pip install poetry
COPY pyproject.toml poetry.lock ./
RUN poetry export -f requirements.txt > requirements.txt
FROM python:3.8-slim
ENV PYTHONUNBUFFERED=1
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD [ "uvicorn", "main:app", "--host", "0.0.0.0" ]
It's also a good idea to set up Docker Compose as well. It may be a good idea to mount the local file for development and then set the auto reload settings.
docker-compose.yml
version: '3'
services:
app:
build: .
volumes:
- ./:/usr/src/app
ports:
- "8000:8000"
command: ["uvicorn", "main:app", "--reload", "--host", "0.0.0.0"]
FAQ
You can use poetry update. However, it will not be updated beyond the version described in pyproject.toml, so please be careful when upgrading major versions.
You can remove it with poetry remove PACKAGE_NAME.
You can install the entire development dependency package with poetry install.
So far, we have explained the development flow of a Python project using Poetry. In Poetry as well as Pipenv, I think that by utilizing poetry.lock, it is possible to eliminate the version difference between developers and provide a stable development environment. It also follows the format agreed upon by PEP, so it feels like it's not bad for the future.
On the other hand, I get the impression that it is a project management tool for ** Python package developers ** rather than Python applications, such as lacking the script function that runs the one-liner that was possible with Pipenv. I use it quite a bit to run Lint and other tests, but this feature.
So both Poetry and Pipenv have their advantages and disadvantages. At this point, I think it's safe to choose the one that suits your project. I hope you all have a good Python life.
Recommended Posts