Note that I'm managing the python environment with Poetry and I want to put different packages for the learning server and the inference server.
Basically, I thought it would be good to plunge everything into dependencies, but it seems that there are powerful and vicious packages in the world that require a few GB of memory just by installing, so even poor infrastructure can install it. Hey ... that became a problem.
It sounds reasonable to (temporarily) strengthen your infrastructure, but are you confident that you will definitely resize your instance after the installation is complete over the next few months or years? I don't. So I devised it.
poetry has the concepts of ʻoptional and ʻextras.
These are options for the ʻadd and ʻinstall subcommands, respectively.
In conclusion, the following steps worked:
$ poetry add --optional SOME_STRONG_PACKAGE
$ vi pyproject.toml
#↓ Add
[tool.poetry.extras]
train = ["torch", "torchtext"]
↑ Group name ↑ List of packages to install (make sure it is optional in dependencies)
$ poetry install
$ poetry install -E train
When I try the above method on a project that is already managed by poetry, I get the following error:
[ValueError]
Extra [train] is not specified.
After that, I deleted poetry.lock and tried the above poetry install -E train again and it worked.
Obviously the error message is unfriendly </ s> PR chance.
Recommended Posts