Build an application with Clean Architecture while using DI + mock in Python

When implementing a web application using Python in business, I decided to introduce Clean Architecture in order to develop while separating concerns for each layer.

We've put together a set of practices for developing testable applications using Dependency Injection, which also serves to share knowledge with team members.

Clean Architecture

This time, the purpose is a sample using Python, so the explanation of Clean Architecture is simple.

CleanArchitecture (1).jpg (Quoted from The Clean Architecture)

Clean Architecture is of interest in Architecture proposed by Robert C. Martin in 2012 A design method for achieving separation. Hexagonal architecture (advocated in 2005) and onion architecture (advocated in 2008) are architectures with the same aim.

The main advantages are clarification of business logic (although it is not unique to Clean Architecture), independence from UI, DB and framework, and improvement of testability.

As written in the original text, it is not necessary to cut the layers according to the concentric circle sample, and you can increase or decrease the layers as needed, but this time I decided to cut the layers according to the textbook.

Gradual typing using typing

Gradual Typing has become established in dynamically typed languages these days, but Python can also benefit from typing during development by using typing.

Unlike TypeScript, type errors are not detected at build time, but with an IDE like PyCharm (IntelliJ IDEA), you can develop while benefiting from the type, so you can achieve a development experience comparable to static typing.

Clean Architecture incorporates the Dependency Inversion Principle, and the use of typing will be essential for achieving operations through the interface.

Dependency injection utilizing abstract classes

Python doesn't have an interface, but it does have abstract classes. Abstract classes can have implementations, but the important thing in the Dependency Inversion Principle is that they depend on abstractions, so abstract classes can also achieve that requirement.

Let's reverse the dependency of the Repository layer that the Usecase layer actually calls through an abstract class.

class SampleRepository(metaclass=ABCMeta):
    @abstractmethod
    async def get(self, resource_id: str) -> dict:
        raise NotImplementedError


class SampleRepositoryImpl(SampleRepository):
    async def get(self, resource_id: str) -> dict:
        return {"id": id}


class SampleUsecase:
    sample_repository: SampleRepository

    def __init__(self, repository: SampleRepository):
        self.sample_repository = repository

   def get(self, resource_id: str) -> dict:
        return asyncio.run(self.sample_repository.get(resource_id))


SampleUsecase(repository=SampleRepositoryImpl()).get()

At this time, the Repository that the Usecase layer knows is an abstract class and does not know its implementation. Also, it is possible to give an implementation to an abstract class, but doing so violates the law of dependency inversion, so by giving @abstractmethod, the concrete class has an implementation.

Dependency Injection is the key to Clean Architecture, but you can inject dependencies with so-called vanilla DI without using a DI container. (Of course, DI container is not unnecessary)

Typing type checking is powerful when injecting dependencies, and with IDE support you should feel little inconvenience with dynamically typed languages.

Unit test using mock

One of the advantages of Clean Architecture is testability. Partial mocking can be done easily, coupled with Dependency Injection. Also, take advantage of the powerful features of the Python standard ʻunit test` module.

`test_sample.py`


class SampleRepositoryMock(SampleRepository):
    async def get(self, resource_id: str) -> dict:
        raise NotImplementedError


class TestSampleUsecase(TestCase):
    def test_get(self):
        get_mock = AsyncMock(return_value={"id": "0002"})

        repository = SampleRepositoryMock()
        repository.get = get_mock
        usecase = SampleUsecase(repository=repository)

        self.assertEqual(usecase.get("0002"), {"id": "0002"})
        get_mock.assert_called_with("0002")

MagicMock is provided as a partial mock, and you can easily replace the method by using this. You can also verify that it was called.

In addition, ʻAsyncMock is also provided as an awaitable return value, which eliminates the need to generate a provisional ʻasync def to return a native coroutine or a low-level Future type variable. .. When communicating to the outside, native coroutines using ʻasync / await` are often used in Python these days, but this can also be handled only by standard modules.

And at the time of testing, Dependency Injection is used to insert a dummy Repository for testing into the Usecase layer and verify the operation of only the layer to be tested.

By the way, the same thing can be achieved by injecting a class instance that is actually used without preparing a dummy class and then replacing it using a partial mock. Which is not the correct answer for this, if you want to focus only on the layer under test, it is better to inject a dummy class, and if you are testing across layers you don't need a dummy.

The reason why it is desirable to be a dummy in the former case is that if you forget the mock, the test will pass depending on the layer on which it depends, and the dummy class that raises an exception if you do not use the mock is more reliable. This is because it can be detected.

Use dataclass in Entity layer

In Clean Architecture, business logic is given to the Entity layer. Entity reminds me of DDD, but it's not exactly the same as Entity in DDD, nor is it the same as Value Object.

According to the original text, "Entity is a set of data structures and functions that encapsulates business rules."

Entities encapsulate Enterprise wide business rules. An entity can be an object with methods, or it can be a set of data structures and functions. It doesn’t matter so long as the entities could be used by many different applications in the enterprise.

(Quoted from The Clean Architecture)

There is no strict regulation on what kind of property the entity should have, but personally, dataclass introduced from Python 3.7 ) I think it goes well with it.

@dataclass
class Article(frozen=True, eq=True)
  id: str
  body: str

article1 = Article(id="1", body="test")
article2 = Article(id="1", body="test")

#True by value-based identity verification(Eq by default=True is set)
article1 == article2 

#An error occurs because it is frozen and is an immutable object.
article1.id = "2"

The functions that dataclass itself has are convenient by themselves, but in particular, properties such as frozen for giving invariants and ʻeq` for identity verification by value are expected behaviors for the Entity layer. (Of course, invariance and value comparison are not always required)

Also, not only in the Entity layer, it is troublesome to define the __eq__ method sequentially when verifying the identity of an object in a unit test, and the merit of using the data class will be great.

Samples have been published on Github

I posted the source of a simple web application that actually works on Github because it would be redundant to put all the sample code in the article.

https://github.com/t-tiger/Python-CleanArchitecture-Example

I'm using Flask as a web framework, but according to the idea of Clean Architecture, only the Rest layer depends on the library, and the rest is almost vanilla Python code.

Unit test code is also included, so check it out if you're interested.

Finally

Python has become one of the most popular languages in recent years as machine learning has grown in popularity, and it seems that its use in the web domain is increasing accordingly.

With the advent of Gradual typing and dataclass, the dynamic typing language has fully covered the areas where the development experience was impaired, and with the addition of traditional productivity, it is also an attractive option for web applications. It has become.

I myself have been writing Python at the production level for a few months, but I hope it will be helpful for those who want to try Python in the future because I can make use of the experience I have cultivated in other languages.