Framework development in Python
agenda
- Self-introduction
- Background
- What is written in this article
- Think about abstraction
- Framework development
- Summary
Self-introduction
Hi, my name is Furuuchi and I am a CTO / server-side engineer at Anyflow Inc.. My usual language is Python, and I also work with the Django framework. Django writes code that looks like this: https://github.com/furuuchitakahiro/django_othello Anyflow is working on the field of iPaaS. iPaaS is an abbreviation of integration Platform as a Service. Specifically, it is a platform for inter-SaaS integration such as integration of Slack and spreadsheets.
background
At Anyflow, the main task of server-side engineers is to integrate various SaaS. For example, implement code that sends a message in Slack or reads a value from a spreadsheet. However, Slack and spreadsheets run on completely different systems, and you need to implement the working code for each SaaS. If you dig deeper, there are many implementations in Slack for sending messages, sending files, getting user information, and so on. It's not so much when it comes to linking all of these, but with the Oreole script, there is no need to wait for the death march. To solve this problem, Anyflow has developed a SaaS integration framework. I will summarize what I have been working on.
What is written in this article
** I try to use the content as a reference for improving the mounting ability. ** **
In this article, we are demonstrating duck typing implementation while capturing abstraction and concreteness. The contents are explained in the order of starting with simple and easy-to-understand ones and gradually applying them, giving examples. Even if you don't develop a framework, you can get a little understanding of frameworks like Django that you usually use.
Think abstract
Have you ever worked on or thought about ** "abstract" **, all of a sudden? I didn't think too much before creating Anyflow.
First of all, I will explain ** human ** as the subject in an easy-to-understand manner. "Taro Tanaka is a human being." If you think of this as abstract and concrete "** Taro Tanaka [Specific] ** is ** Human [Abstract] **." Will be. This level is easy, isn't it? By the way, let's convert it to Python.
class Human:
def __init__(self, name: str):
self.name = name
if __name__ == '__main__':
taro = Human(name='Taro Tanaka')
print(taro.name)
# > 'Taro Tanaka'
It's really easy. This alone does not reveal the benefits of abstraction, so we will delve into it. If you are a human being, you have an age in addition to your name, you can do greetings, and you have a family. Based on these, we will convert it to Python again.
from typing import Iterable, List
class Human:
def __init__(self, name: str, age: int):
self.name = name
self.age = age
def greeting(self) -> str:
return f'I{self.name}My name is. Age is{self.age}I'm old.'
class Family:
def __init__(self, members: Iterable[Human]):
self.members = list(members)
def all_greeting(self) -> List[str]:
return [member.greeting() for member in self.members]
if __name__ == '__main__':
taro = Human(name='Ichiro Tanaka', age=20)
jiro = Human(name='Jiro Tanaka', age=19)
saburo = Human(name='Saburo Tanaka', age=18)
tanaka_family = Family(members=[taro, jiro, saburo])
print(tanaka_family.all_greeting())
# > [
# 'My name is Ichiro Tanaka. The age is 20 years old.',
# 'My name is Jiro Tanaka. I am 19 years old.',
# 'My name is Saburo Tanaka. I am 18 years old.'
# ]
Suddenly the code became longer. The point to look at is the ** Family ** class. The Family class takes an array of Human class instances in its constructor. The logic at which the all_greeting method works assumes that members are a Human class instance in the list. From a different point of view, ** members can be regarded as okay if only the condition **, which is an array of Human class instances, is satisfied. (So-called generics) This is the result of combining the human-like abstraction of the Human class with the family abstraction of the Family class, which is a collection of human beings. Extreme story This implementation is ** any name and age for humans, and even if there are 100, it will function as a family if it is human **. I didn't touch on any specifics, but focused on human characteristics and family characteristics.
It has become clear that abstraction is the definition of the common denominator (likeness) of things.
Framework development
From here on, it's important to develop the framework. What I explain is so-called ** duck typing **. On Wikipedia ** [If it walks like a duck and sounds like a duck, it must be a duck](https://ja.wikipedia.org/wiki/%E3%83%80%E3% 83% 83% E3% 82% AF% E3% 83% BB% E3% 82% BF% E3% 82% A4% E3% 83% 94% E3% 83% B3% E3% 82% B0) ** It has been.
This commentary creates the following functions based on something like ** FaceBook ** focusing on your hometown.
- name --Age
- Birthplace ――Answer to the standard question of the specialty of your hometown
The concept is ** a FaceBook **-like guy with a profile of what he thinks of his hometown specialties. We'll name this ** "Hometown FB" ** for illustration purposes. Now let's make it Python code. The configuration is as follows.
--main.py (main processing) --mods (module directory) --human.py --questions.py --prefectures.py (prefectures)
First, implement the questions in questions.py.
from typing import ClassVar
class BaseQuestion:
QUEASTION: ClassVar[str] = ''
def __init__(self, answer: str):
self.answer = answer
@property
def to_qa(self) -> str:
return f'Q. {self.__class__.QUEASTION}\nA. {self.answer}'
class SeaQuestion(BaseQuestion):
QUEASTION: ClassVar[str] = 'Is the sea of your hometown beautiful?'
class FujisanQuestion(BaseQuestion):
QUEASTION: ClassVar[str] = 'Is Mt. Fuji beautiful?'
BaseQuestion defines questionability. Question-likeness means having the concept and answer of an interrogative sentence. SeaQuestion and FujisanQuestion define specific questions for this BaseQuestion. The important thing here is that we haven't touched on the answer yet. ** A question is regarded as a question sentence and an answer sentence, and on top of that, there is a concrete and standard question sentence, and there are multiple answer sentences about it **. The point is that if you inherit from BaseQuestion, you can treat them all equally as questions. All questions have a function that allows you to spit out question sentences and answer sentences together as a character string. (to_qa method) In other words, the BaseQuestion class acts as a questioning framework. Inheriting BaseQuestion and asking a lot of questions will be the job of my hometown FB.
Then implement the prefectures in prefectures.py.
from typing import ClassVar, Tuple, Iterable
from . import questions
class BasePrefecture:
NAME: ClassVar[str] = ''
QUESTIONS: ClassVar[Tuple[questions.BaseQuestion, ...]] = tuple()
def __init__(self, qa_list: Iterable[questions.BaseQuestion]):
self.qa_list = list(qa_list)
@classmethod
def get_questions(cls) -> Tuple[str, ...]:
return tuple(question.QUEASTION for question in cls.QUESTIONS)
class Tokyo(BasePrefecture):
NAME: ClassVar[str] = 'Tokyo'
QUESTIONS: ClassVar[Tuple[questions.BaseQuestion, ...]] = (
questions.SeaQuestion
)
class Sizuoka(BasePrefecture):
NAME: ClassVar[str] = 'Shizuoka'
QUESTIONS: ClassVar[Tuple[questions.BaseQuestion, ...]] = (
questions.SeaQuestion,
questions.FujisanQuestion
)
class Yamanashi(BasePrefecture):
NAME: ClassVar[str] = 'Yamanashi'
QUESTIONS: ClassVar[Tuple[questions.BaseQuestion, ...]] = (
questions.FujisanQuestion
)
This is the same structure as the question. The BasePrefecture class only defines that the state has a name and a question. Inherit the defined BasePrefecture class and define the Tokyo class and so on. At this time, set the Tokyo class-ness (name and question list). The most specific state is the state where you have a place of origin and a set of questions and answers. The BasePrefecture class acts as a prefectural framework.
Finally, human.py is a human implementation.
from . import prefectures
class Human:
def __init__(
self, name: str, age: int, prefecture: prefectures.BasePrefecture
):
self.name = name
self.age = age
self.prefecture = prefecture
def greeting(self) -> str:
qa_list_str = '\n'.join([qa.to_qa for qa in self.prefecture.qa_list])
return (
f'I{self.name}My name is. Age is{self.age}I'm old.'
'\nQ&A'
f'\n{qa_list_str}'
)
The Human class is almost the same. It's almost the same as [Thinking about abstraction](#Thinking about abstraction). As an additional point, prefecture is added to the constructor, and greeting (self-introduction) is just added to the question and answer of that prefecture.
Click here for the main.py implementation of how to use
from mods import questions, prefectures
from mods.human import Human
if __name__ == '__main__':
tokyo = prefectures.Tokyo(qa_list=[
questions.SeaQuestion(answer='It's a little dirty.')
])
taro = Human(name='Taro Tanaka', age=20, prefecture=tokyo)
print(taro.greeting())
# >My name is Taro Tanaka. The age is 20 years old.
# Q&A
# Q.Is the sea of your hometown beautiful?
# A.It's a little dirty.
shizuoka = prefectures.Sizuoka(qa_list=[
questions.SeaQuestion(answer='Is beautiful.'),
questions.FujisanQuestion(answer='its big.')
])
jiro = Human(name='Jiro Tanaka', age=22, prefecture=shizuoka)
print(jiro.greeting())
# >My name is Jiro Tanaka. I am 22 years old.
# Q&A
# Q.Is the sea of your hometown beautiful?
# A.Is beautiful.
# Q.Is Mt. Fuji beautiful?
# A.its big.
yamanashi = prefectures.Yamanashi(qa_list=[
questions.FujisanQuestion(answer='its big.')
])
saburo = Human(name='Saburo Tanaka', age=24, prefecture=yamanashi)
print(saburo.greeting())
# >My name is Saburo Tanaka. I am 24 years old.
# Q&A
# Q.Is Mt. Fuji beautiful?
# A.its big.
Implementation summary
It may be too minimal to look like a framework, but I think it touches on the essentials. The nice thing about duck typing is that the code is easy to see. By properly separating the particle size (abstraction) of responsibility of the class, you can clearly express the problem you want to solve and maintain extensibility (diversity).
There are only three base codes for this hometown FB: BaseQuestion, BasePrefecture, and Human classes. Over time, the hometown FB service will be expanded by inheriting BaseQuestion and BasePrefecture.
When you are doing web development, I think you are making XX models using frameworks. Of course, at this time, you inherited and declared the framework model class, right? The framework model class only defines the behavior for reading, creating, updating, and deleting, and doesn't mention anything about columns. I think that columns are defined in the XX model class you create. The relationship between the model class of the framework and the model you define is that of duck typing.
This demonstration was a business-specific framework. Implementing a more abstract thing (model view controller) according to the contents of this article makes it a general-purpose framework for web application development.
Summary
Actually, Python has a lot of contents to tell when developing more frameworks such as ABC, docstring for documentation, type hints and DI, but since it is a sentence amount that one book can write. This time, I made a demonstration that mainly considers abstraction and concreteness and puts it into duck typing.
** Anyflow is 10,000 times more intense than the content of this article! ** ** ** I have a lot of things to talk about, such as development history and using Airflow! ** ** ** If you like or are interested in logic and design, please let us know! ** ** ** For those who are interested in launching services as well as engineering at startups! ** **
Anyflow has a lot of momentum! I won the B Dash Camp! https://jp.techcrunch.com/2019/10/31/b-dash-camp-2019-fall-pitch-arena/ I won the Incubate Camp! https://jp.techcrunch.com/2019/09/14/incubate-camp-12th/
Recruitment page: https://open.talentio.com/1/c/anyflow/requisitions/detail/13828 If you are interested in more casual, please DM to https://www.facebook.com/takahiro.furuuchi.37!
Recommended Posts