I made a python text

● Text body (PDF format 299 pages: 3.12MB) Updated on 2020/05/14 (→ DL from mirror site) Main libraries mentioned:  Kivy, argparse, socket, threading, concurrent, requests,  BeautifulSoup, mpmath, subprocess, datetime, pickle, struct,  base64, collections, enum, shutil, zipfile, wave, PyAudio,  Tkinter → Sample program in the text __ [How to learn] __ Run the sample while reading Chapters 1 and 2. I think it is good to understand while doing so. (You can skip the parts that you find difficult) Read the other chapters as needed. Make your own variation of the execution example in the text You will deepen your understanding if you proceed while trying. For those who already know Python to some extent, grammar and examples I think it can also be used to find out. ● Library book (PDF format 185 pages: 3.41MB) Updated on 2020/06/03 (→ DL from mirror site) (Old name: Python3 module book) Library mentioned:  OpenCV, Pillow, pygame, Eel, PyDub, NumPy, matplotlib,  SciPy, SymPy, hashlib, passlib, Cython, Numba, ctypes,  PyInstaller, JupyterLab, json, urllib, zenhan, jaconv → Sample program in library book Although NumPy and matplotlib are extremely sophisticated I'm worried about how much I should remember. So The amount of content that "If you remember it for the time being, it will be easier later" I'm looking for it. --------------------- Introducing seminar activities using Python. → Introduction of activities applying Python --------------------- I'm writing an article because I want people to think calmly about the strengths / weaknesses of Python in case some people blindly believe in Python. → Consideration of the advantages and disadvantages of Python --------------------- [Useful link] ・ Compiled library for Windows It is a format to be installed by PIP. This is useful when the standard installation method does not work. (Since it is "Unofficial", please do so at your own risk.) --------------------- [Data processing related] I am making a text about the basic knowledge for data processing in Python. It is still a prototype, and I would like to hear your opinions and advice. → Text of data processing by Python --------------------- 【PySimpleGUI】 Probably the easiest GUI library for Python to use. I would like to hear your opinions as well. → Basic usage of PySimple GUI ---------------------   (2017/05/16) I teach programming at school. I made an introductory text for Python 3 to teach beginners. It's a stance of an introductory book for those who have already mastered some language to quickly get used to Python.   It explains at once to the point where you can create an application using the GUI library "Kivy". (Introduction to Kivy) Also, an introductory explanation of requests and BeautifulSoup4 for web scraping is provided. (Basic explanation about Tkinter)   Although the degree of completion is still low, the text will be revised and updated if it is used in the field of education. We will publish it for free, so we would be grateful if you could give us your opinion. If you find any mistakes, please let us know. Contact information is on the back cover of the text.

Introduction page   Those who want to use this are welcome. I would like to improve the degree of perfection while receiving opinions. Especially, I hope it will be used by teachers who teach programming at school. We would appreciate it if you could give us any requests or suggestions from the field. (Reflect in the text as much as possible) If it is within the scope of this book, you can also send us your questions. We will do our best to accommodate you.

We also distribute Video Teaching Materials that summarizes the contents of the text.

Supplement (updates, information, etc.)

-------------- [Supplement](2020/05 / 27-06 / 03) Added to the library book. I added various additions to the explanation about the handling of NumPy and SymPy matrices. I added an explanation of how to draw a pie chart and a horizontal bar graph with matplotlib. I added (how to specify the header) to the explanation part of the CSV output of the NumPy array. Other minor modifications   -------------- [Supplement](2020/04 / 05-05 / 14) Added to the text body. I added a little to the part of split assignment. Added a commentary on float calculation error. Added a commentary on the zipfile module for handling ZIP archives. The most basic explanation about pprint is posted. Some additions to the regular expression. We have posted a sample to convert a number with a decimal point to a binary number / reverse conversion. Added a commentary on the chain of comparison operators. Some additions to the appendix "Management of memory usage". Other minor corrections (I find embarrassing mistakes every time I check ...)   Added to the library book. Added an explanation of how to draw a 3D bar graph with matplotlib. Also, some additions to the explanation of zero matrix creation (NumPy). The file I / O method was added to the explanation of the json module. Some additions to the min, max, argmin, and argmax parts of NumPy. I explained how to plot polar coordinates with matplotlib. In the sympy module section, we have included a description of the var function and a set of functions related to prime numbers.   (Apology) There was an error in the explanation about LaTeX output in the sympy section of the library book. Please replace it with the latest version as it has been corrected.   -------------- [Supplement](2020/03 / 13-24) Added to the text body. Added explanation of how to output a file with the print function. Added a commentary on how to write a tuple of variables after the for statement and perform split assignment. Added explanation about collections.Counter. Added a commentary on issubclass to check the inheritance relationship of classes. Added regarding the output of WAV files. Other additions and corrections

Added to the library book. Explains how to aggregate NumPy arrays uniquely. Explains how to insert and delete elements in NumPy arrays.   -------------- [Supplement](2019/12 / 01-20) Added to the text body. Added (re.sub, re.split) to the regular expression. I added a commentary on splitlines that splits a character string that spans multiple lines. The explanation about the assignment formula (Python3.8 ~) is included. Added a description of the argparse module for command argument parsing. A rudimentary explanation about the enum module is included. Other minor corrections and additions.   -------------- [Supplement](2019/09 / 01-14) Some additions and corrections to the text body. We have posted a sample program that enumerates all Unicodes in which characters are defined.

-------------- [Supplement](2019/08 / 01-31) Added to the text body. -Addition of basic explanation about assert and raise that raises an exception -Added to the explanation about dictionary objects -Addition of explanation about bit operation -Addition of explanation about shutil module ・ Other additions and corrections

You have pointed out (or scolded me) about the notation of the "" symbol. Since the text is written in TeX, it is difficult to put out the half-width "Japanese yen symbol", and there were many parts that were written using the full-width "¥". As you pointed out, this is a very bad thing, so I'll fix it. → (It has been fixed considerably)

Added to the library book. An example of the procedure for drawing a 3D scatter plot with matplotlib is posted. I also started writing about SciPy. Other additions and corrections. __Correction and apology __: There is an error in the sample program scatter3d01.py published in the 7/31 version of the library book. Please replace it with the latest version as it has been corrected.   -------------- [Supplement](2019/06 / 08-07 / 26) Added to the text body. A sample program to investigate how much the calculation speed changes depending on whether or not NumPy is used is included in the appendix. Also, I added an explanation of how to suppress the display of warning messages and a content that introduces memory-profiler that checks the memory usage of the Python processing system. These are required when processing data with Jupyter Lab.   Some additions to the NumPy part of the library book. I explained how to apply the high speed of matrix operation and how to handle images as array data. We also added a description of the table function that draws a two-dimensional array as a table with matplotlib. __Correction and apology __: In the explanation of NumPy's FFT, "amplitude spectrum" was mistakenly described as "power spectrum". I'm sorry. I have corrected it, so please replace it.

-------------- [Supplement](2019/05 / 21-06 / 06) The name "module book" may not be good. There are still errors in the handling of terms such as "module", "package", and "library" in the contents of the distributed text series. I started thinking about fixing it. Change the name to "Python3 Library Book".

According to the Ministry of Education, Culture, Sports, Science and Technology website, Python is used for programming in high school information departments. It is written. I feel that the content of the text to be distributed needs to be considered very carefully.

-------------- [Supplement](2019/05 / 01-18) Added to the text body. I added a commentary on enumerate. The more you learn Python, the more you realize that it is a treasure trove of tricks for ease of use.

Added to the module book. I added the part of NumPy. The basic usage of the PyDub module is posted. In addition, some additions related to OpenCV and PyGame. It's almost time to check (overhaul) the module book as a whole (please wait)

-------------- [Supplement](2019/04 / 06-30) Added to the text body. Since f-string is a very convenient function, I have included an explanation of its basic usage. Also, added a brief explanation about Anaconda's conda command. Added to the module book. I added an explanation of how to draw a heat map with matplotlib and how to convert an image into a numerical array of pixels. In addition, some explanations of NumPy have been added and modified.   -------------- [Supplement](2019/02 / 22-03 / 28) Maintenance of the text body.

Kivy is recommended for making "decent apps".   -------------- Supplement Since it became necessary to execute window drawing and keyboard & mouse handling at high speed, I started to summarize how to use Pygame.   I will add it to the module book soon.   -------------- Supplement Added to the text body. Added the method to check whether the key exists in the dictionary in the explanation part of the dictionary type object. Also added about data type conversion. I realized that the explanation of the surprisingly rudimentary matters was missing. After all, when you actually start using the text, you will find cracks one after another. Text body   -------------- Supplement The introduction explanation of hashlib is included in the module book. Module Book (Cryptographic hash generation module) It is necessary for concealing password character strings and creating digital signatures for documents.   -------------- Supplement I added the "Generate list using for" part of the text body, and also added the conversion between characters and character codes. Text body   When students start using it, the parts to be added will appear one after another.   -------------- Supplement "Disclaimer" was added to each text.   -------------- Supplement I added a little bit to the text itself. (Part of subprocess) Also, I added a commentary on how to concatenate character strings. (I missed the explanation though it is an important method) Text body   -------------- Supplement The index of the text itself was poor, so I added a little bit. Text body   -------------- Supplement I received a question (request?) "Would you like to explain the pandas module?"   please wait for a while. I also have only one body ... However, I was worried about it, so I may start it around the summer vacation. I will actually need it for my seminar activities ... (I have no promise when it will be)   It seems that pandas is useful for statistical analysis.   -------------- Supplement I started to write numpy and matplotlib in the module book. Module Book   From now on, it seems that the number of additions to the module book will increase with the activities of the seminar.   -------------- Supplement Added a method to start an external program using the subprocess module. Text body It also explains how to start multiple external programs and pipe I / O asynchronously with them.   Regarding the asyncio module, it seems that it goes beyond the scope of "beginner" and "introduction", so I am thinking of making it a commentary content in a form different from this text. (Slight policy change)   After that, if you put an introductory explanation of the numpy and matplotlib modules in the module book, it seems that you can use it as a teaching material for seminars for a while.   Also, I added a method for generating random numbers. (Indispensable for creating games)   -------------- Supplement Numpy is a standard for high-speed numerical operations in Python. However, numpy is not universal in every aspect of the research subject, and in order to obtain a numerical solution of an arbitrarily set differential equation, it is necessary to write a simulation program individually.   Regarding numpy, I will write an introductory explanation in the module book, but I have to deal with the theme of my seminar as soon as possible, so I will write the simulation code in C language for a while.   So, we will call the program (executable file) written in C from Python and throw only the part that requires speed to C. It seems that many modules call external programs like that.   So, I'm going to put together the usage of the subprocess and asyncio modules as soon as possible and incorporate them into the text body.   -------------- Supplement Kivy's KV language, I have to organize information for beginners. I thought about how to learn and how to consult with students.   It's the next theme. After a while, I will correct and add the text again ...   -------------- Supplement-2 Students made a "disappointing voice" when Kivy's text input did not support the Japanese input method of Windows / Mac. But it's still in the future.   The display during input may be corrupted, but for the time being, I can input Japanese somehow ... Kivy is just developing. He persuaded the students that "the UI has the highest potential and will be the focus of attention in the future!" I have personal expectations, but I will be dating Kivy for a while.

Among them, I think there is no doubt that the update will greatly improve it.   -------------- Supplement It seems that the Japanese translation of the Kivy document site is in progress. Kivy Document Site   Thank you. This is useful when you need a detailed reference.   -------------- Supplement The introduction explanation of SymPy is included in the module book. Module Book (Computer algebra module) You can do things like Mathematica and Maxima.   You may be wondering "Why SymPy?", But the theme of our seminar was "Proposal of a user interface for an easy-to-use computer algebra system", and that's the situation.   SymPy is a great package, and it's a level that competes with commercial systems except for graphical features (probably). The manual also has more than 1,700 pages in PDF ...   I think Maxima, which is free to use as a formula calculator, or Mathematica, if you have money, is good. You can easily make 3D plots. (I personally don't recommend forcing SymPy to be a mathematical calculator as it is. I think it's a hassle ...)   I think SymPy will be very useful when developing apps that incorporate computer algebra functions (educational software, thinking support apps, etc.).   -------------- Supplement About Kivy: Added explanation of basic usage of ActionBar (Kivy style menu), ScreenManager, Carousel, TabbedPanel. (Responded to requests from students)   Also, I added a basic explanation about the Clock module required to handle time events (timers) in Kivy. There was a student who tried to use for and while to repeat the process of starting at the timing in the GUI application. Instead, set a timer.   With this, I wonder if I can give guidance on making apps ...   I'm worried about cracking, but I'll crush it over time. Text body If you have any problems, please contact us. We will correct and upload it immediately. -------------- Supplement-2 When making an app with Kivy, using the KV language makes debugging a little difficult for beginners ...   If you make a mistake in a Python program, the error message is easy to understand, but it is difficult to detect the mistake made in KV ... In short, the message "There is a XX error in what line of the KV language" does not appear.   For experts, the KV language is appreciated in terms of development efficiency, but for beginners, the KV language is likely to reduce the efficiency of studying ...   Until you get used to Kivy, I think it's better to write APIs in Python normally, but what do you think ...   -------------- Supplement We asked you about the handling of Kivy's frame buffer (FBO), so we added a commentary. It seems that it is difficult to extract the pixel value with ordinary Canvas drawing. That's why we use FBO. Text body   -------------- Supplement The introduction explanation of Pillow is included in the module book. Module Book (Image processing module) You can do things like Photoshop.   -------------- Supplement The release date is July, but I immediately received a voice saying "I want you to teach me how to use the sound", so I added the content. (Wave, PyAudio) In addition, we have made some additions and corrections in response to consultations from students.   I started to make a "module book". I wrote only the part like the introduction of OpenCV module. (Image recognition module) Module Book   As for modules, there are too many types published, so I think I will mainly write about the ones that I found out how to use in my seminar.

Udemy related

It is quite difficult for those who have graduated (or are enrolled) in the Faculty of Humanities (Humanities) to learn programming by themselves. The Python texts that are released for free assume that those who are already familiar with other languages will learn Python in a short time.   If you are learning programming for the first time, you will need a detailed guide on what to do specifically. Those who are actually educated about the Python language at school and those who have a certain number of friends who are familiar with Python can naturally understand the detailed points in learning, but it is not in such an environment. I think that there are some disadvantages in learning.   I actually teach programming, and I strongly feel about this. Those who have mastered C ++, Java, and JavaScript to some extent can learn by themselves only with Python texts, but those who do not can get stuck early with texts alone. For such a person, I have to give a detailed explanation at the operation level in front of the PC.     Nowadays, many excellent teaching materials (books, internet sites, etc.) related to Python are sold and released, including paid and free ones, and I think that it is becoming less difficult to obtain information sources. .. However, I think that it is difficult to obtain the "detailed points of operation" mentioned above from the text medium, and for the time being, I am thinking of dealing with this with video teaching materials.   So far, we are delivering one video course on Udemy, and I would like to introduce it here. It is a more compact summary of the free text "Introduction to Python 3".   Please understand that this is a video course that is completely unnecessary for those who can learn Python only with the free text "Introduction to Python 3", other books, and information on Internet sites.   This video material from Udemy was created for those who have been frustrated by self-study in Python or who are learning programming for the first time in Python. (11 hours in total)   >>> Link to Course   ・ Facebook ・ Twitter

table of contents

[Table of contents of the text body]

1 Introduction --- 1 1.1 What you can do with Python --- 1 1.2 Contents of this manual --- 1 1.2.1 How to read this book --- 2 1.3 How to install and start the processing system --- 2 1.3.1 Python processing distribution (distribution form) --- 2 1.3.2 Starting Python processing system --- 2 1.3.3 Interactive mode --- 3 1.4 GUI library to be used --- 3 1.5 Detailed information about Python --- 3 2 Basics of Python --- 4 2.1 Script execution --- 4 2.1.1 Comments to be written in the program --- 5 2.1.2 Program indent --- 5 2.2 Variables and data types --- 6 2.2.1 Variable release (discard) --- 7 2.2.2 Numerical value --- 7 2.2.2.1 Maximum value, minimum value --- 10 2.2.2.2 Rounding of floating point numbers --- 11 2.2.2.3 Mathematical functions --- 11 2.2.2.4 Special values: inf, nan --- 12 2.2.2.5 Handling of floating point numbers with multiple precision --- 14 2.2.2.6 Handling of fractions --- 15 2.2.2.7 Random number generation --- 17 2.2.3 Character string --- 18 2.2.3.1 Escape sequence --- 18 2.2.3.2 raw character string (raw string) --- 19 2.2.3.3 Character string over multiple lines --- 19 2.2.3.4 Decomposition and composition of character strings --- 20 2.2.3.5 Character string replacement --- 21 2.2.3.6 Character replacement --- 22 2.2.3.7 Judgment of letters and numbers --- 22 2.2.3.8 Uppercase / lowercase conversion and judgment --- 23 2.2.3.9 Character string content inspection --- 23 2.2.3.10 Character code / character type --- 24 2.2.4 Truth value (bool type) --- 26 2.2.5 Null object: None --- 26 2.2.6 Type conversion --- 26 2.2.7 Type inspection --- 26 2.2.8 Cardinal conversion --- 27 2.2.8.1 N-ary number → decimal number --- 27 2.2.8.2 Decimal number → n-ary number --- 27 2.2.9 Bit operation --- 28 2.3 Data structure --- 30 2.3.1 List --- 30 2.3.1.1 Access to list elements --- 30 2.3.1.2 Editing the list --- 30 2.3.1.3 Realization of stack and queue by list: pop method --- 32 2.3.1.4 Inspection of list --- 33 2.3.1.5 Exception handling --- 34 2.3.1.6 Counting the number of elements --- 35 2.3.1.7 Alignment of elements (1): sort method --- 36 2.3.1.8 Sorting of elements (2): sorted function --- 37 2.3.1.9 Reversing the order of elements --- 37 2.3.1.10 Duplicate list --- 37 2.3.2 Tuple --- 38 2.3.2.1 Special tuple --- 39 2.3.3 set --- 39 2.3.3.1 Operation of set theory --- 40    2.3.3.2 frozenset --- 41  2.3.4 Dictionary type --- 41 2.3.4.1 Access to the dictionary by get method --- 41 2.3.4.2 How to retrieve the key and value columns --- 42 2.3.4.3 How to check the number of elements in a dictionary type object --- 42 2.3.4.4 How to generate a dictionary from a list or tuple --- 43 2.3.4.5 How to retrieve all elements of a dictionary as columns --- 43 2.3.5 Advanced application of subscripts (slices) --- 43 2.3.5.1 Slice object (reference) --- 44 2.3.6 Data structure conversion --- 45 2.3.7 Value allocation according to the data structure (split assignment) --- 46 2.3.7.1 Selective partial extraction of data structure --- 47 2.3.8 Data structure shuffle --- 47 2.3.9 About the speed of access to the data structure --- 47 2.4 Control structure --- 48 2.4.1 Repeat (1): for --- 48 2.4.1.1 range object --- 49 2.4.1.2 else in the for statement --- 50 2.4.1.3 Data structure generation using for (element inclusion notation) --- 50 2.4.1.4 Iterator --- 51 2.4.1.5 For statement using split assignment --- 52 2.4.1.6 zip function and zip object --- 52 2.4.1.7 Addition of index information by enumerate --- 53 2.4.2 Repeat (2): while --- 54 2.4.3 Repeated interruption and skip --- 55 2.4.4 Conditional branch --- 55 2.4.4.1 Conditional expression --- 56 2.4.4.2 Conditional judgment regarding various "empty" values --- 56 2.4.4.3 Comparison by ʻis' --- 58 2.5 Input / output --- 59 2.5.1 Standard output --- 59 2.5.1.1 Formatting of output data --- 59 2.5.1.2 Handling of standard output by sys module --- 61 2.5.2 Standard input --- 62 2.5.2.1 Handling of standard input by sys module --- 63 2.5.3 Input from file --- 64 2.5.3.1 Handling of byte strings --- 67 2.5.3.2 How to check the code system of byte string --- 68 2.5.3.3 How to read the specified number of bytes --- 68 2.5.3.4 How to read the contents of the file at once --- 68 2.5.3.5 How to read a file as an iterator --- 68 2.5.4 Output to file --- 69 2.5.5 Standard error output --- 69 2.5.6 Handling of paths (files, directories) --- 71 2.5.6.1 Operations related to the current directory --- 71 2.5.6.2 List of directory contents --- 72 2.5.6.3 Get file size --- 72 2.5.6.4 File and directory inspection --- 72 2.5.6.5 Delete files and directories --- 72 2.5.6.6 Information about running scripts --- 73 2.5.6.7 Concatenation of path expressions --- 73 2.5.6.8 Yet another way to handle paths: pathlib --- 73 2.5.7 Acquisition of command arguments --- 75 2.5.8 Be careful when processing input / output --- 76 2.6 Function definition --- 77 2.6.1 Formal parameters --- 77 2.6.1.1 Function with an indefinite number of arguments --- 77 2.6.1.2 How to write `*'in the argument when calling a function --- 78 2.6.1.3 Keyword argument --- 78 2.6.2 Variable scope (handling of variables inside and outside the function definition) --- 79 2.6.3 Deletion of defined function --- 80 2.7 Object-oriented programming --- 81 2.7.1 Class definition --- 81 2.7.1.1 Constructor --- 81 2.7.1.2 Method definition --- 81 2.7.1.3 Class variables --- 82 2.7.1.4 Attribute investigation (property investigation) --- 84 2.7.2 Investigation of class inheritance relationship --- 84 2.8 Programming according to the data structure --- 85 2.8.1 map function --- 86 2.8.1.1 Map of a function that takes multiple arguments --- 87 2.8.1.2 How to give a zip object to the map function --- 87 2.8.2 lambda and function definition --- 88   2.8.3 filter --- 90  2.8.4 if ~ else as a ternary operator ... --- 90 2.8.5 Batch judgment by all, any --- 90 2.8.6 Higher-order function module: functools --- 92    2.8.6.1 reduce --- 92  2.9 Substitution formula --- 93 3 Building a GUI application with Kivy --- 94 3.1 Kivy Basics --- 94 3.1.1 Implementation of application program --- 94 3.1.2 Concept of GUI construction --- 94 3.1.2.1 Widget --- 95 3.1.2.2 Layout --- 96 3.1.2.3 Screen --- 96 3.1.3 Handling of windows --- 97 3.1.4 Disable multi-touch --- 97 3.2 Basic GUI application construction method --- 98 3.2.1 Event processing (introduction) --- 98 3.2.1.1 Event handling --- 98 3.2.2 Example of application construction --- 99 3.2.3 Event processing (method by registering callback) --- 102 3.2.4 Registering and deleting widgets --- 103 3.2.5 Handling of application start and end --- 104 3.3 How to use various widgets --- 105 3.3.1 Label: Label --- 105 3.3.1.1 Registering fonts in resources --- 107 3.3.2 Button: Button --- 108 3.3.3 Text input: TextInput --- 108 3.3.4 Checkbox: CheckBox --- 109 3.3.5 Progress Bar: ProgressBar --- 109 3.3.6 Slider: Slider --- 109 3.3.7 Switch: Switch --- 109 3.3.8 Toggle Button: ToggleButton --- 109 3.3.9 Image: Image --- 110 3.3.9.1 Sample program --- 110 3.4 Canvas Graphics --- 111 3.4.1 Graphics class --- 112    3.4.1.1 Color --- 112     3.4.1.2 Line --- 112     3.4.1.3 Rectangle --- 112     3.4.1.4 Ellipse --- 113  3.4.2 Sample program --- 113 3.4.2.1 Sine function plot --- 113 3.4.2.2 Display of various figures and images --- 113 3.4.3 Drawing to the frame buffer --- 115 3.4.3.1 Pixel value retrieval --- 116 3.4.3.2 Coordinate position obtained from the event --- 117 3.5 ScrollView --- 117 3.5.1 Widget size setting --- 118 3.6 Setting to fix the window size (prohibit resizing) --- 118 3.7 Construction of UI in Kivy language --- 119 3.7.1 Basics of Kivy language --- 119 3.7.1.1 Explanation using a sample program --- 119 3.7.1.2 Correspondence between Python programs and Kv files --- 121 3.8 Event by time --- 122 3.8.1 Time event schedule --- 122 3.9 GUI construction format --- 122 3.9.1 Screen handling: Screen and Screen Manager --- 123    3.9.1.1 ScreenManager --- 123     3.9.1.2 Screen --- 123  3.9.2 Action Bar: ActionBar --- 125 3.9.3 Tab panel: TabbedPanel --- 126 3.9.4 Swipe: Carousel --- 127 4 Matters necessary for practical application development --- 129 4.1 Date and time processing --- 129 4.1.1 Basic functions --- 129 4.1.1.1 Date and time formatting --- 130 4.1.1.2 datetime property --- 130 4.1.2 Use of time module --- 131 4.1.2.1 Time measurement --- 131 4.1.2.2 Waiting for program execution --- 132 4.2 Character string search and regular expression --- 132 4.2.1 Pattern search --- 132 4.2.1.1 Search using regular expressions --- 134 4.2.1.2 Union of search patterns --- 136 4.2.1.3 Pattern matching using regular expressions --- 137 4.2.1.4 Pattern matching at the beginning and end of the line --- 137 4.2.2 Replacement process: re.sub --- 138 4.2.2.1 Replacement process over multiple lines --- 138 4.2.2.2 Replacement process with reference to pattern matching group --- 139 4.2.3 Application to character string decomposition: re.split --- 139 4.3 Multi-thread and multi-process --- 140 4.3.1 Multi-thread --- 140 4.3.2 Multi-process --- 141    4.3.2.1 ProcessPoolExecutor --- 141  4.3.3 Comparison of execution time between multithread and multiprocess --- 142 4.4 Generator --- 143 4.4.1 Generator function --- 143 4.4.2 Generator type --- 144 4.5 Split programming by creating modules and packages --- 145 4.5.1 Module --- 145 4.5.1.1 Module as a single source file --- 145 4.5.2 Package (library configured as a directory) --- 146 4.5.2.1 Module execution --- 147 4.5.3 Investigation of the directory where the module is located --- 148 4.5.4 About \ _ \ _ init \ _ \ _. Py --- 148 4.6 Random access in the file --- 150 4.6.1 Specifying the access location of the file (seek of the file) --- 150 4.6.2 Sample program --- 150 4.7 Saving and loading data objects: pickle module --- 152 4.8 Binary data creation and deployment: struct module --- 154 4.8.1 Creation of binary data --- 154 4.8.2 Expansion of binary data --- 154 4.8.3 Byte order --- 156 4.9 How to convert binary data to text: base64 module --- 157 4.10 exec and eval --- 159 4.10.1 Namespace designation --- 159 4.10.2 eval function --- 159 4.11 collections module --- 160 4.11.1 Queue: deque --- 160 4.11.1.1 Adding and extracting elements: append, pop --- 160 4.11.1.2 Rotation of element order: rotate --- 161 4.11.2 Aggregation of elements: Counter --- 161 4.11.2.1 Extract the aggregated results in order of appearance frequency --- 162   4.11.3 namedtuple --- 162  4.12 Enumeration type: enum module --- 163 4.12.1 Enum type --- 163 4.12.2 Example of how to handle constants --- 164   4.12.3 IntEnum --- 165  4.12.4 Assigning a value to the Enum element by the auto function --- 165 4.13 Exception (error) handling --- 167 4.13.1 How to raise an exception --- 167 4.14 Investigation of symbols used --- 169 4.15 with syntax --- 170 4.16 Formatted display of data structure: pprint module --- 175 4.17 Acquisition of information on processing environment --- 176 4.17.1 Acquisition of Python version information --- 176 4.17.2 Use of platform module --- 176 4.17.3 Reference of environment variables --- 177 4.18 Operations on files and directories: shutil module --- 179 4.18.1 Duplicate files and directories --- 179 4.18.2 Handling of archive files (archive) and compression processing --- 180 4.19 Handling of ZIP archive: zipfile module --- 181 4.19.1 Open the ZIP archive file --- 181 4.19.2 Addition of members to the archive --- 181 4.19.3 Confirmation of the contents of the library --- 181 4.19.4 Reading archive members --- 182 4.19.5 Development of library --- 182 4.19.5.1 Access to password-protected ZIP archive --- 183 4.20 Handling of command arguments: argparse module --- 184 4.20.1 Command argument format --- 184 4.20.2 How to use --- 184 4.20.2.1 Setting optional arguments --- 185 4.20.2.2 Position argument setting --- 185 4.20.2.3 Command line analysis processing --- 185 4.20.3 Explanation according to the sample program --- 185 4.20.3.1 Help function --- 186 4.20.3.2 Specifying the command argument type --- 187 4.20.4 Subcommand implementation method --- 187 4.21 End of script (end of program) --- 189 5 TCP / IP communication --- 190 5.1 socket module --- 190 5.1.1 Preparation of socket --- 190 5.1.2 Server-side program processing --- 191 5.1.3 Client-side program processing --- 191 5.1.4 Send and receive --- 191 5.1.5 Sample program --- 191 5.2 WWW content analysis --- 193 5.2.1 requests library --- 193 5.2.1.1 Method related to request transmission --- 193 5.2.1.2 Method related to acquired content --- 194 5.2.1.3 Assess based on Session object --- 195 5.2.2 Beautiful Soup Library --- 195 5.2.2.1 Handling of HTML content in BS --- 196 6 Cooperation with external programs --- 198 6.1 How to start an external program --- 198 6.1.1 Standard input / output connection --- 198 6.1.1.1 Closing standard input of external program --- 201 6.1.2 Asynchronous input / output --- 201 6.1.3 Synchronization with external process (waiting for termination) --- 203 6.1.4 An easier way to start an external program --- 203 7 Sound input / output --- 205 7.1 Basic knowledge --- 205 7.2 WAV format file input / output: wave module --- 205 7.2.1 Opening and closing WAV format files --- 205 7.2.1.1 Various attributes of WAV format data --- 206 7.2.2 Reading from WAV format file --- 206 7.2.3 Sample program --- 206 7.2.4 Relationship between the number of quantization bits and the sampling value --- 207 7.2.5 Handling of read frame data --- 207 7.2.6 Example of outputting WAV format data (1): From list to WAV file --- 209 7.2.7 Example of outputting WAV format data (2): From NumPy array to WAV file --- 210 7.2.8 Notes on sound data size --- 211 7.3 Sound input and playback: PyAudio library --- 212 7.3.1 Sound input / output via stream --- 212 7.3.2 Playing WAV format sound files --- 213 7.3.2.1 Detection of end of sound playback --- 215 7.3.3 Input from a voice input device --- 215 A Information about Python --- 219 A.1 Python internet site --- 219 A.2 Example of Python installation work --- 219 A.2.1 PSF version installation package method --- 219 A.2.2 Method by Anaconda --- 220 A.3 Mechanism of starting Python --- 220 A.3.1 Starting PSF version Python --- 220 A.3.2 Starting Anaconda Navigator --- 221 A.3.3 Starting Anaconda Prompt --- 222 A.3.3.1 Management of Python environment by conda command --- 223 A.4 Library management by PIP --- 223 A.4.1 Solution when PIP command cannot be executed --- 224 B Information about Kivy --- 225 B.1 Example of Kivy installation work --- 225 B.2 Information for avoiding troubles when using Kivy --- 225 B.2.1 Setting of drawing API used by Kivy --- 225 B.2.2 About SDL --- 226 B.3 GUI design tool --- 226 C Tkinter: Basic GUI toolkit --- 227 C.1 Basic handling --- 227 C.1.1 Usage example --- 228 C.1.1.1 Setting whether to change the window size --- 229 C.1.2 Widget placement --- 229 C.2 Various widgets --- 231 C.2.1 Check button and radio button --- 231 C.2.1.1 Variable class --- 232 C.2.2 Entry (text box) and combo box --- 233 C.2.2.1 End of program --- 235 C.2.3 List box --- 235 C.2.4 Text (character editing area) and scroll bar --- 236 C.2.5 scale (slider) and progress barber --- 238 C.2.5.1 Callback function setting of Vaviable class --- 239 C.3 Menu construction --- 239 C.4 Canvas drawing --- 241 C.4.1 Drawing method (partial) --- 241 C.4.2 Figure management --- 244 C.5 Event handling --- 246 C.5.1 Execution of a function with a specified time --- 248 C.6 Display of multiple windows --- 249 C.7 Acquisition of information about displays and windows --- 250 C.8 Messagebox --- 251 C.8.1 Handling of application termination --- 253 D Library handling --- 254 D.1 Thing about reading of library --- 254 D.1.1 Granting an alias when reading the library --- 254 D.1.2 Reading the library to omit the prefix --- 254 D.1.2.1 Precautions when omitting the prefix (name conflict) --- 255 D.2 Introduction of various libraries --- 256 E Ingenuity to make the interactive mode easier to use --- 257 E.1 Suppression and display of warning messages --- 257 E.2 Memory usage status management --- 258 F sample program --- 259 F.1 Comparison of list / set / dictionary access speed --- 259 F.1.1 Access in the form of giving an integer index to the slice --- 259 F.1.2 Time required for membership inspection --- 260 F.2 Comparison of calculation speed with and without library use --- 263 F.3 pathlib application example --- 266 F.4 Conversion between floating point number and binary number --- 267 F.5 List of all Unicode characters --- 268

[Table of contents of library book]

1 Image input / output and processing --- 1   1.1 OpenCV --- 1  1.1.1 Video input --- 2 1.1.2 User interface --- 2 1.1.2.1 Input from video file --- 3 1.1.3 Saving the frame to a file --- 3 1.1.4 Reading still images --- 4 1.1.5 Color separation and composition --- 5 1.1.6 Inspection of image similarity --- 6 1.1.6.1 Calculation of AKAZE features --- 6 1.1.6.2 Collation of feature data --- 6 1.1.6.3 Sample program --- 7   1.2 Pillow --- 9  1.2.1 Loading and saving image files --- 9 1.2.1.1 Resolution when reading EPS --- 10 1.2.2 Creating a new Image object --- 10 1.2.3 Viewing images --- 11 1.2.4 Editing images --- 11 1.2.4.1 Image enlargement / reduction --- 11 1.2.4.2 Extraction of the image part --- 12 1.2.4.3 Image duplication --- 12 1.2.4.4 Paste image --- 12 1.2.4.5 Image rotation --- 12 1.2.5 Image processing --- 13 1.2.5.1 Color separation and composition --- 13 1.2.6 Drawing --- 13 1.2.7 Creating an animated GIF --- 15 1.2.8 Conversion from Image object to numerical array of pixels --- 15 2 GUI and multimedia --- 16   2.1 pygame --- 16  2.1.1 Basic matters --- 16 2.1.1.1 Surface object --- 16 2.1.1.2 Application execution loop --- 16 2.1.2 Drawing function --- 18 2.1.2.1 Drawing sample program --- 21 2.1.2.2 Rotation, scaling sample program --- 22 2.1.3 Keyboard and mouse handling --- 24 2.1.4 Audio playback --- 25 2.1.4.1 Method using Sound object --- 26 2.1.4.2 Conversion from Sound object to NumPy array --- 27 2.1.5 Use of sprites --- 29   2.2 Eel --- 34  2.2.1 Basic matters --- 34 2.2.2 Calling a function written in Python / JavaScript --- 36 2.2.2.1 About the return value of the function --- 37 2.3 Media data conversion: PyDub --- 38 2.3.1 Reading audio files --- 38 2.3.2 Conversion from voice data to NumPy array --- 38 2.3.3 Conversion from NumPy array to voice data --- 39 2.3.4 Saving audio files --- 39 2.3.4.1 Save as MP3 format file --- 39 2.3.4.2 Save as WAV format file --- 39 3 Science and technology --- 40 3.1 Library for numerical calculation and visualization: NumPy / matplotlib --- 40 3.1.1 Generation of array object --- 40 3.1.1.1 About the element type of the array --- 41 3.1.1.2 Array of truth values --- 41 3.1.1.3 Special values: infinity and non-number --- 41 3.1.1.4 Data string generation (sequence generation) --- 42 3.1.1.5 Generation of multidimensional array --- 43 3.1.1.6 Investigation of the shape of the array --- 43 3.1.1.7 Access to array elements --- 44 3.1.1.8 Operation when a data string is given to a slice --- 45 3.1.1.9 Access to specified rows and columns --- 45 3.1.1.10 Deformation of array shape --- 45 3.1.1.11 Row and column transposition --- 47 3.1.1.12 Row, column inversion and rotation --- 47 3.1.1.13 Type conversion --- 48 3.1.2 Array concatenation and repetition --- 48 3.1.2.1 Concatenation by append and concatenate --- 48 3.1.2.2 Connection by hstack and vstack --- 49 3.1.2.3 Repeating array by tile --- 50 3.1.3 Expansion of array dimensions --- 50 3.1.3.1 Newaxis object method --- 50 3.1.3.2 Method by expand \ _dims --- 51 3.1.4 Data extraction --- 51 3.1.4.1 Masking by truth value sequence --- 51 3.1.4.2 Extraction of elements by conditional expression --- 52 3.1.4.3 Combined conditional expressions by logical operators --- 52 3.1.4.4 Extraction and replacement of elements by where --- 52 3.1.4.5 Search for maximum value, minimum value, and its position --- 53 3.1.5 Data sorting (sorting) --- 53 3.1.5.1 Two-dimensional array alignment --- 54 3.1.5.2 How to get the index of the alignment result --- 54 3.1.6 Various processing for arrays --- 55 3.1.6.1 Elimination of duplicate elements --- 55 3.1.6.2 Aggregation of integer elements --- 55 3.1.7 Calculation for data string: 1D to 1D --- 56 3.1.8 Data visualization (basic) --- 56 3.1.8.1 Basic procedure of drawing process --- 57 3.1.8.2 Two-dimensional plot: Line graph --- 57 3.1.8.3 Creating multiple graphs --- 59 3.1.8.4 Matplotlib graph structure --- 62 3.1.8.5 Display of Japanese headings / labels --- 66 3.1.8.6 How to save the graph as an image file --- 67 3.1.9 Random number generation --- 68 3.1.9.1 Generation of uniform random numbers --- 68 3.1.9.2 Generation of integer random numbers --- 68 3.1.9.3 Generation of normal random numbers --- 68 3.1.9.4 Random number seed --- 68 3.1.9.5 RandomState object --- 69 3.1.10 Statistical processing --- 70 3.1.10.1 Total --- 70 3.1.10.2 Maximum value, minimum value --- 70 3.1.10.3 mean, variance, standard deviation --- 70 3.1.10.4 Percentage points --- 71 3.1.10.5 Section and tabulation (class and frequency survey) --- 71 3.1.10.6 Data shuffle --- 72 3.1.11 Data visualization (2) --- 73 3.1.11.1 Histogram --- 73 3.1.11.2 Scatter plot --- 73 3.1.11.3 Bar graph --- 74 3.1.11.4 Box plot --- 74 3.1.12 Calculation on data string: \ (n ) dimension to 1 dimension --- 75 3.1.13 Data visualization: 3D plot --- 76 3.1.13.1 Wireframe --- 76 3.1.13.2 Surface plot --- 77 3.1.13.3 3D scatter plot --- 78 3.1.14 Data visualization: Other --- 79 3.1.14.1 Heat map --- 79 3.1.14.2 Creating a table --- 81 3.1.15 Fast Fourier Transform (FFT) --- 83 3.1.15.1 Conversion from time domain to frequency domain: Fourier transform --- 83 3.1.15.2 Conversion from frequency domain to time domain: Fourier transform / inverse transform --- 83 3.1.15.3 Specifying the aspect ratio of the plot --- 86 3.1.15.4 Precautions when using Fourier transform --- 87 3.1.16 Complex number calculation --- 87 3.1.16.1 Square root of complex number --- 87 3.1.16.2 Complex number norm --- 87 3.1.16.3 Conjugate complex number --- 87 3.1.16.4 Complex number argument (phase) --- 88 3.1.17 Matrix calculation --- 89 3.1.17.1 Matrix sum and product --- 89 3.1.17.2 Identity matrix, zero matrix, etc. --- 89 3.1.17.3 Make all matrix elements the same value --- 89 3.1.17.4 Diagonal component, diagonal matrix --- 90 3.1.17.5 Transpose of matrix --- 91 3.1.17.6 Determinant and inverse matrix --- 91 3.1.17.7 Eigenvalues and eigenvectors --- 91 3.1.17.8 Matrix rank --- 92 3.1.17.9 Complex conjugate matrix --- 92 3.1.17.10 Hermitian conjugate matrix --- 92 3.1.17.11 Vector norm --- 93 3.1.18 I / O: Array object file I / O --- 94 3.1.18.1 Saving to a text file --- 94 3.1.18.2 Reading from a text file --- 94 3.1.18.3 I / O to binary files --- 95 3.1.18.4 Compressed storage and loading of data --- 95 3.1.19 Matrix comparison --- 96 3.1.20 Implementation of user-defined functions that process arrays --- 96 3.1.21 Example of improvement of calculation speed by applying matrix calculation --- 98 3.1.22 Handling of image data --- 99 3.1.22.1 Conversion from pixel array to image data (linkage with PIL library) --- 100 3.1.22.2 Conversion from image data to pixel array (linkage with PIL library) --- 101 3.1.22.3 Sample program: Three-color separation of images --- 101 3.1.23 Other functions --- 103 3.1.23.1 Extraction of data above / below the standard --- 103 3.1.23.2 Extraction of data in the specified range --- 104 3.2 Library for scientific calculation: SciPy --- 107 3.2.1 Signal processing tool: scipy.signal --- 107 3.2.1.1 Generation of basic waveform --- 107 3.3 Computer algebra library: SymPy --- 111 3.3.1 Precautions regarding loading of modules --- 111 3.3.2 Basic matters --- 111 3.3.2.1 Object for computer algebra --- 112 3.3.2.2 Simplification of mathematical formulas (evaluation) --- 112 3.3.2.3 Structure of expression {\ tt f (x, y, ...) (extracting head and argument sequence) --- 113 3.3.2.4 Constant --- 113 3.3.3 Basic computer algebra function --- 114 3.3.3.1 Expansion of formula --- 114 3.3.3.2 Factorization --- 114 3.3.3.3 Arrangement by specified symbols --- 114 3.3.3.4 Approximate minutes: Simplification of fractions (1) --- 115 3.3.3.5 Partial fractions --- 115 3.3.3.6 Simplification of fractions (2) --- 116 3.3.3.7 Substitution (replacement of symbols) --- 116 3.3.3.8 Various mathematical functions --- 116 3.3.3.9 Formula type --- 116 3.3.4 Analytical processing --- 117 3.3.4.1 Extreme --- 117 3.3.4.2 Derivative function --- 117 3.3.4.3 Delayed execution of differential operation --- 118 3.3.4.4 Primitive function --- 118 3.3.4.5 Delayed execution of integrate --- 118 3.3.4.6 Definite integral --- 119 3.3.4.7 Series expansion --- 119 3.3.5 Solving various equations --- 119 3.3.5.1 Solving algebraic equations --- 119 3.3.5.2 Solving differential equations --- 120 3.3.5.3 Solving the difference equation (difference equation, recurrence formula) --- 121 3.3.6 Linear algebra --- 121 3.3.6.1 Determinant --- 121 3.3.6.2 Inverse matrix --- 121 3.3.6.3 Eigenvalues, eigenvectors --- 122 3.3.7 Summation --- 122 3.3.8 Pattern matching --- 123 3.3.9 Numerical calculation --- 124 3.3.9.1 Prime factorization --- 124 3.3.9.2 Approximate value --- 124 3.3.10 Format conversion output --- 124       3.3.10.1 LaTeX  --- 124  3.3.11 Graph plot --- 124 3.3.11.1 How to save the graph to an image file --- 126 4 Security-related --- 127   4.1 hashlib --- 127  4.1.1 Basic usage --- 127   4.2 passlib --- 127  4.2.1 Algorithms that can be used --- 127 5 Acceleration of programs / application construction --- 129   5.1 Cython --- 129  5.1.1 Usage example --- 129 5.1.2 Adjustment for speeding up --- 131   5.2 Numba --- 132  5.2.1 Basic usage --- 132 5.2.2 Speeding up by specifying the type --- 133   5.3 ctypes --- 133  5.3.1 Example of creating a shared library in C language --- 134 5.3.2 Example of calling a function in a shared library --- 134 5.3.3 Handling of arguments and return values --- 135 5.3.3.1 Transfer of array data --- 137   5.4 PyInstaller --- 140  5.4.1 Simple usage example --- 140 5.4.1.1 How to build as a single executable file --- 141 6 Dialogue work environment (JupyterLab) --- 142 6.1 Basic matters --- 142 6.1.1 Start and end --- 143 6.1.2 Display area configuration and operation method --- 143 6.1.2.1 Notebook usage example --- 144 6.1.2.2 About the kernel --- 146 6.1.3 Execution of input function in Notebook --- 146 6.2 Comment display by Markdown --- 147 6.3 Usage example --- 148 6.3.1 Formatted display of SymPy formulas by MathJax --- 148 6.3.2 Sound playback with IPython.display module --- 149 7 Others --- 151 7.1 json: Use of data exchange format JSON --- 151 7.1.1 JSON notation --- 151 7.1.2 Usage example --- 151 7.2 urllib: URL processing --- 152 7.2.1 Handling of other byte characters (`%'encoding) --- 152 7.2.1.1 Character code system specification --- 152 7.3 zenhan: Full-width ⇔ half-width conversion --- 153 7.4 jaconv: Various conversions related to Japanese characters --- 153


written by Katsunori Nakamura

Recommended Posts

I made a python text
I made a Line-bot using Python!
I made a daemon with Python
〇✕ I made a game
I made a payroll program in Python!
I made a character counter with Python
I made a Hex map with Python
After studying Python3, I made a Slackbot
I made a roguelike game with Python
I made a simple blackjack with Python
I made a configuration file with Python
I made a neuron simulator with Python
I made a competitive programming glossary with Python
I made a weather forecast bot-like with Python.
I made a GUI application with Python + PyQt5
I made a Twitter fujoshi blocker with Python ①
[Python] I made a Youtube Downloader with Tkinter.
I made a Caesar cryptographic program in Python.
I made a bin picking game with Python
I made a Mattermost bot with Python (+ Flask)
I made a Python Qiita API wrapper "qiipy"
I made blackjack with python!
I made a discord bot
I made blackjack with Python.
I made wordcloud with Python.
I made a Twitter BOT with GAE (python) (with a reference)
I made a prime number generation program in Python
I made a login / logout process using Python Bottle.
I made a net news notification app with Python
I made a VM that runs OpenCV for Python
I made a Python module to translate comment outs
I made a Python3 environment on Ubuntu with direnv.
I made a LINE BOT with Python and Heroku
[Python] I made a classifier for irises [Machine learning]
I made a prime number generation program in Python 2
I made a python library to do rolling rank
I made a repeating text data generation tool "rpttxt"
I made a C ++ learning site
I made a CUI-based translation script (2)
I made a wikipedia gacha bot
I made my own Python library
I made a CUI-based translation script
I made a simple typing game with tkinter in Python
I made a package to filter time series with python
[VSCode] I made a user snippet for Python print f-string
I made a quick feed reader using feedparser in Python
I made a simple book application with python + Flask ~ Introduction ~
I made a puzzle game (like) with Tkinter in Python
I made a Chatbot using LINE Messaging API and Python
[Python] I made a Line bot that randomly asks English words.
I made a new AWS S3 bucket
I made a simple circuit with Python (AND, OR, NOR, etc.)
I made a dash docset for Holoviews
I made a library to easily read config files with Python
[Python3] I made a decorator that declares undefined functions and methods.
I made a package that can compare morphological analyzers with Python
I made a script in Python to convert a text file for JSON (for vscode user snippet)
I made Othello to teach Python3 to children (4)
I touched "Orator" so I made a note
Python> I made a test code for my own external file
I drew a heatmap with seaborn [Python]