I created a template for a Python project that can be used universally

I use Python a lot, but I used to hit the same code every time to build the environment. This takes a lot of time & the parts used are the same, so I published the repository as a project template that can be used for general purposes. https://github.com/odrum428/python_setup

Purpose

When I started a Python project, I always entered the same commands, such as around CI and documents.

It's like initializing a project using Pipenv, defining a CI, putting in tests and lint, and documenting with Sphinx.

It took me a while to get these in place each time, so I created a project template with the aim of configuring it for any Python project.

It is made to be compatible with analysis, application, and model generation.

Configuration overview

package

Pipenv is used for package and environment management. The following articles are organized for these management tools, so I think you should refer to them. Understanding best practices around Python packages

At present, it is best to use pipenv and manage various settings with setup.py and setup.cfg. It feels like I've used it, and it's much easier to manage and build.

The packages used in the project created this time are summarized.

• isort
• flake8
• Sphinx
• pytest
• sphinx-rtd-theme

As a basic lint system, we have introduced ʻisort that arranges package import etc. and flake8that arranges code style. In addition to this, there are alsoyapf etc., but I adopted the standard PEP8 and made it a comfortable flake8`. Here is the code of the error part It is set to be done.

We adopted pytest for the test. To be honest, if you use pytest for python testing, I think that there is no problem. There are many things that pytest can do than standard unittest, and it is easy to handle.

In addition, this template uses Sphinx to generate documents. The documentation is generated from the test code and docstring defined in the tests folder. By generating documents only from test cases, we aim to actively write test cases and enrich the documents. The theme of the document is sphinx-rtd-theme.

CI We have also introduced CI using Circle CI. Run Lint with ʻisort and flake8to keep the code style, and run the test code withpytest. Tox, which enables multiple versions of test execution, and pytest-randomly`, which makes tests random, are excluded because they are not used for general purposes.

In addition to these, we have also implemented automatic documentation updates. When there is an update in the tests folder, I update the document with CI and even commit Git.

  is_docs_update:
    steps:
      - run:
          name: check tests folder is updated
          command: |
            if [[ ! $(git diff --diff-filter=d --name-only HEAD | grep tests/ ) = '' ]]; then
              echo "build and deploy"
            else
                echo "no need docs update"
                circleci step halt
            fi
      - run:
          name: make rst file
          command: |
            pipenv run sphinx-apidoc -f -o docs/ tests/
      - run:
          name: make html
          command: |
            pipenv run sphinx-build -a ./docs ./docs/public
      - run:
          name: git push
          command: | 
            git config --global user.email [email protected]
            git config --global user.name odrum428
            git add -A
            git commit -m 'updating docs [skip ci]'
            git push origin HEAD

file organization

The file structure is shown below.

 ├ .circleci/
 ├ .envrc
 ├ .github/
 ├ .gitignore
 ├ docs/
 │  ├ Makefile
 │  ├ conf.py
 │  ├ index.rst
 │  ├ make.bat
 │  ├ modules.rst
 │  ├ public/
 │  └ tests.rst
 │
 ├ src/
 ├ tests/
 ├ LICENSE
 ├ Pipfile
 ├ Pipfile.lock
 ├ README.md
 ├ setup.cfg
 └ setup.py

I easily separated the structure into source, test, and document.

App code, machine learning, analysis code, etc. are all managed in src in a nice way. Write the test code with the same file structure. Documentation is generated from the test code. It is an image like.

From here, I think we should expand it according to the application.

Packages and various settings are done in setup.cfg. This greatly simplifies the code and can be written as:

setup.py

from setuptools import setup

setup()

The actual contents of the settings are set in setup.cfg.

[metadata]
name = sample-package
version = '1.0'
auther = Keita Mizushima
auther_email = [email protected]
description = sample repogitory of python
description-file = file: README.md
url = http://example.com
license = MIT
license_file = LICENSE

[options]
install_requires=
packages = find:

[flake8]
show_source = True
max-line-length=120
max-complexity=15

How to use

Simply register a new project based on this repository.

1. Create a GitHub repository.

This time I created a repository called new_project.

2. Start a new project based on this repository

git clone [email protected]:odrum428/python_setup.git new_poject
cd new_project
git remote set-url origin [email protected]:user_name/new_project.git
git push origin master

Now you can create a new project while taking over this repository.

3 Install pipenv.

pip install pipenv

As a bonus, direnv automatically switches between virtual environments, so it's recommended! You don't have to activate the environment every time. https://github.com/direnv/direnv

Summary

I will continue to update it every time the environment changes, so please use it if you have trouble with the configuration. Also, I would be grateful if you could give us a PR to the repository.

https://github.com/odrum428/python_setup