[LINUX] I want to tell people who want to import from a higher directory with Python direnv

The problem you want to solve in this article

I decided to organize the folders of python files that had been placed properly until now, as the project became larger. The problem that occurred here was that I could not import my own library. In python, only the execution folder and the folders under it are the path to be searched for modules, so you cannot import from files higher than yourself.

Until now, I used sys.path.append to import from the upper file, but it has the following disadvantages and I am wondering if there is a way to solve it without using sys.path.append. did.

Weaknesses of sys.path.append

When I try to import from a higher level file using sys.path.append, the first few lines of the file always look like this:

import sys
import pathlib

sys.path.append(str(pathlib.Path(__file__).resolve().parents[1]))
from MyModule.MyClass import MyClass

What I hate about this is

--You have to sys.path.append all the files you want to import from the higher level file --When using an automatic code formatting tool such as autopep8, the import statement is moved up without permission.

That is the point. It's inevitably contrary to the ideal of writing clean code.

Direnv solves such a problem this time. direnv is a tool that defines environment variables in .envrc for each directory and enables environment variables only when the directory where .envrc is located becomes the current directory.

Writer's development environment

OS：macOS Catalina 10.15.6 brew 2.5.2 bash : 3.2.57(1)-release (x86_64-apple-darwin19) Python : 3.6.10

Folder structure used in this explanation

grand_mother
┣ mother
┃　┗ me.py
┗ aunt
　　┗ cousin.py 

The explanation is based on the assumption that cousin.py is imported and used in me.py. me.py and cousin.py are as follows, respectively.

cousin.py

class Cousin:
    name = 'Reon'
    age = 14

me.py

from aunt.cousin import Cousin
print('name :', Cousin.name)
print('age :', Cousin.age)

Install direnv

For Mac, one brew command.

$ brew install direnv

If you're not on a Mac, git clone it.

$ cd /path/to/directory #Where you want to install
$ git clone https://github.com/direnv/direnv
$ cd direnv
$ sudo make install

Add a hook to bash.

~/.bashrc

eval "$(direnv hook bash)"

Reload ~ / .bashrc to get direnv to work.

$ source ~/.bashrc

Make direnv available

Go to the top of the project (in this case grand_mother /) where you want to set the environment variables and run the following command:

$ cd 〇〇/grand_mother
$ direnv edit .

Then grand_mother / .envrc will be created, so add PYTHONPATH.

grand_mother/.envrc

export PYTHONPATH="〇〇/grand_mother:$PYTHONPATH"

Now you can do things like from mother import and from aunt import in the files in grand_mother.

What is PYTHONPATH?

PYTHONPATH adds a directory to look for modules in addition to the default python modules when python loads the modules. You can check the currently set PYTHONPATH with the following command.

#Check PYTHON PATH
$ echo $PYTHONPATH

#Show all environment variables including PYTHON
$ printenv | grep PYTHON

Also, if you want to see all the directories that python loads when loading modules, you can check in python's sys.path.

import sys
print(sys.path)

(Only for those who need it) If you want to write environment variables in .env

• When using direnv, it is easier to write environment variables in .env than in .envrc

Using this article as a reference, I will also explain how to write environment variables in the direction of grand_mother / .env. I chose this method because I had already written other environment variables in .env.

That's it for .envrc.

grand_mother/.envrc

dotenv

Let's add the following sentence to .env.

grand_mother/.env

PYTHONPATH=/〇〇/grand_mother/

Check the operation of direnv

After setting .envrc

direnv: error .envrc is blocked. Run `direnv allow` to approve its content.

You may see an error statement like this. At that time, as shown in the error statement, execute direnv allow and confirm that PYTHONPATH has been added. This sentence is displayed every time you rewrite .envrc.

$ direnv allow
direnv: loading ~/〇〇/grand_mother/.envrc
direnv: export +PYTHONPATH

If you exit the directory where .envrc is located, direnv will be temporarily disabled.

$ cd ..
direnv: unloading

If you enter the directory where .envrc is located again, you will see that the contents of .envrc have been added to the environment variables.

$ cd grand_mother/
direnv: loading .envrc

solved

Now running me.py works fine.

$ python mother/me.py
name : Reon
age : 14

Operation of .envrc

When sharing source code on GitHub etc., .env and .envrc contain information that should be stolen. Or you may need to set different values depending on the execution environment, so try not to commit to the repository by adding .envrc to .gitignore.

(Supplement) Introduce direnv to Amazon Linux

This time, I had to put direnv in the Amazon Linux that I was operating according to my local, so I will write down the work contents at that time.

Amazon Linux environment

$ cat /etc/os-release
Amazon Linux AMI 2018.03
$ bash --version
GNU bash,Version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

First, Amazon Linux doesn't include go to run direnv, so you need to install it.

$ sudo yum install golang -y

Then git clone and install direnv as shown at the top of this article.

$ cd /path/to/directory #Where you want to install
$ git clone https://github.com/direnv/direnv
$ cd direnv
$ sudo make install

Add GOPATH and direnv hooks to bash. GOPATH specifies the location to build, so you can use any location.

~/.bashrc

export GOPATH=$HOME/go

eval "$(direnv hook bash)"

Let's reload ~ / .bashrc and check the setting result.

$ source ~/.bashrc
$ echo $GOPATH
/home/ec2-user/go

After that, direnv can be used on Amazon Linux by the method of ** Making direnv available ** described at the top of the article.

Reference article

• How to set direnv to manage environment variables on Mac
• How to use direnv
• What is PYTHONPATH
• Use direnv
• How to import parent / sibling hierarchy module in python
• Setting environment variables in local environment in python