I added a function to CPython (build & structure grasp)

1.First of all

This section describes the CPython build and CPython file structure required to add functionality to CPython. This article uses Python 3.10.

• This article was written as a report of EEIC (Department of Electrical and Electronic Engineering / Department of Electronic and Information Engineering, Faculty of Engineering, University of Tokyo) 3rd year experiment "Exploring Large-Scale Software".
• The following articles were written by team members.

I added a function to CPython (ternary operator) Add two types of switch statements to python Pre-increment ()

2. What is CPython?

CPython is Python written in C language, and when it is generally called Python, it often refers to this Python. The code is managed on GitHub and you can also publish a PR.

3. Build CPython

The Official Article is very helpful for building CPython. First, clone the code from the CPython 3.10 repository.

$ git clone https://github.com/python/cpython

Then change directories and run the configure script. Here, -g includes the "debug symbol" in the executable and -O0 is the lowest level of optimization. Also, the folder to install is specified by adding --prefix.

$ cd cpython
$ CFLAGS="-O0 -g" ./configure  --with-pydebug   --prefix=(Installation directory)

Next, compile and install.

$ make -s -j2
$ make install

Now you can install CPython safely. To run the installed Python:

$cd (installed directory)/bin
$ ./python3

4. Added features to CPython

The Checklist for adding features to CPython is officially summarized. In the following, we will explain with reference to this.

4.1 CPython file structure

These are the four most important files to make changes when adding functionality to CPython. I will look at them in order.

4.1.1 python.gram First, let's talk about python.gram. It is a file in Grammer / python.gram of CPython. python.gram defines the Python grammar. Write the syntax to add in this file.

$ make regen-pegen

It will automatically generate a Parser.

4.1.2 python.asdl Next, I will explain about python.asdl. A file for creating an abstract syntax tree (AST). If you have defined a new function in python.gram, you also need to write the function definition in this file.

$ make regen-ast

AST is automatically generated.

4.1.3 compile.c Next, I will explain about compile.c. A file that converts AST to bytecode. Describe the processing for each syntax in bytecode. When adding a grammar, describe the processing of that grammar in bytecode here.

4.1.4 ceval.c Next, I will explain about ceval.c. Execute bytecode. If the existing bytecode cannot be implemented, add a new bytecode here.

5. Summary

So far, we have explained how to build and understand the structure of CPython. CPython has a huge amount of code, but I felt that the amount of code that needed attention was small because the procedure for changing the grammar was organized in the official checklist.

Reference material

-Exploring large-scale software
This is an experimental site.

-CPython
CPython repository

-24. Changing CPython ’s Grammar
A checklist for adding features to CPython was included to help you understand the structure of CPython.

-1. Getting Started
It was helpful for building CPython.