A road to intermediate Python

Introduction

Do you guys program in Python? For those who have some understanding of the basic grammar of Python but want to learn more, here are some intermediate techniques and writing methods.

If you master this article, may you become an intermediate Python user from tomorrow? So let's get started.

About variable assignment

In Python, it is possible to define multiple variables on one line.

When substituting normally:

x = 10
y = 12
z = 35

print(x, y, z)  # 10 12 35

When substituting in one line:

x, y, z = 10, 12, 35

print(x, y, z)  # 10 12 35

As mentioned above, it is possible to define variables in one line. Defining all variables on one line reduces readability, but defining variables that have some relevance increases readability on one line.

Also, when assigning the same value to multiple variables, it is also possible to write as follows. This is an assignment method, but be careful with mutable (variable) objects. For details, please refer to shiracamus's detailed explanation in the comment section of this article.

x = y = z = 10

print(x, y, z)  # 10 10 10

Swapping variables

When replacing the contents of variables, I think that it is common to use temp variables etc., but in the case of Python it is easier to write.

When exchanging variables normally:

x, y = 10, 23
tmp = x
x = y
y = tmp
print(x, y) # 23 10

When swapping variables in one line:

x, y = 10, 23
x, y = y, x
print(x, y) # 23 10

I'm surprised that variables can be exchanged in this way. ..

Variable expansion

In Python, arrays, dictionaries, etc. can be expanded by using the * (asterisk) symbol.

test = [1, 2, 3]

print([test, 4, 5])  # [[1, 2, 3], 4, 5]
print([*test, 4, 5])  # [1, 2, 3, 4, 5]

The code above uses an asterisk to expand and assign an array. If you do not use the asterisk, the array will be assigned in a nested state.

For dictionaries, use two asterisks.

test = {'apple': 'red', 'remon': 'yellow'}

print({**test,'cherrie': 'pink'}) # {'apple': 'red', 'remon': 'yellow', 'cherrie': 'pink'}

Comparison of true and false

I think that there are many cases where the IF statement determines whether a variable is 0 or None. However, it is not pisonic code to write the logic that actually determines the boolean value.

Pythonic means "how to write simple and easy-to-read code like Python"

When judging 0 normally:

test = 0

if test == 0:
    print('NG')
else:
    print('OK')

How to write Python-like conditions:

test = 0

if not test:
    print('NG')
else:
    print('OK')

In pionic code, what is judged to be false is decided in advance, so we do not explicitly write the judgment logic for the decided one. The following are judged to be false. Built-in truth value judgment

Compare with None, is operator

Python comparison operators include == and ʻis`, both of which can be used to check if the variables to be compared are the same. However, if you do not understand the meaning of both, you may cause unexpected bugs, so be careful.

Q. Should I use is for comparison with None? A. Yes. But unless you know what you're doing, you'll get the same results as using ==

First, the operation of the is operator will be explained. The is operator checks if the variables being compared have the same ID. What is the ID of a variable is the reference destination (pointer) of the variable. For example, if the reference destination value is the same but the reference destination (pointer) is different, the is operator determines that it is false.

If the variables are referenced the same:

test = 'my name is taro' + '.'
test2 = test
print(id(test)) # 140643210401160
print(id(test2)) # 140643210401160

In the above case, test2 is assigned a variable called test, so test and test2 point to the same reference. So, when comparing with the is operator, test and test2 are judged to be the same. Let's check the ID of None as well (ID may vary depending on the environment)

test = None
test2 = None
print(id(test)) # 9107296
print(id(test2)) # 9107296

None refers to the same object no matter where it is defined. So, when comparing whether it is None, you can use the is operator to check whether the comparison target is really None. In the case of the == operator, you can change the behavior of the operator itself by using a function called operator overload, so it may not be possible to determine exactly whether it is None or not. (Without operator overloading, the None comparison will have the same result with either the is operator or the == operator.)

Q. Shouldn't we use is for comparisons other than None? A. Yes. Using is for non-None comparisons can lead to unexpected bugs.

In particular, it is dangerous to use the is operator a lot because it may behave unexpectedly when comparing strings of variables. For example, consider the following code.

test = 'my name is taro' + '.'
test2 = 'my name is taro' + '.'
print(id(test)) # 140196053288256
print(id(test2)) # 140196053288328

print(test) # my name is taro.
print(test2) # my name is taro.

if test is test2:
    print('same')
else:
    #This is output
    print('not same')
    
if test == test2:
    #This is output
    print('same')
else:
    print('not same')

Although test and test2 contain the same character string, they are referenced differently. So, when comparing with the is operator, the two variables are judged to be different, and the == operator is judged to be the same. The == operator has the disadvantage of being slower than the is operator because it checks and judges the contents in the variable, but the use of the is operator is confusing for comparisons other than None. So I don't recommend it. However, in exceptional cases, the is operator can be used to write code neatly. (See righteous's comment in the comments section)

Short-circuit evaluation

Other languages have a short-circuit evaluation method, but Python also has one. By making good use of short-circuit evaluation, it is possible to write the logic of IF statements smartly.

For example, consider the following code.

test = {'age': 20}

if 'age' in test:
    if test['age'] >= 20:
        print('I'm an adult')

First, the first IF statement checks if the test variable has an attribute called age. This is to prevent an error if you access an attribute that does not exist. The second IF statement confirms whether the value of the age attribute is 20 or more.

Now let's rewrite it using short-circuit evaluation.

test = {'age': 20}

if 'age' in test and test['age'] >= 20:
    print('I'm an adult')

You might think that you simply connected the IF statements with the and operator. However, if you swap the left and right of the and operator, the behavior you mean will change. The exact behavior of short-circuit evaluation (and operator and or operator) is that if the first condition is true (or operator) or false (and operator), the next processing is not performed. Become. In the above case, if the evaluation result of 'age' in test is false, test ['age']> = 20 will not be evaluated. Therefore, it is possible to achieve the condition that test ['age']> = 20 is evaluated only when the age attribute exists in the test variable.

Combination of comparison operators

I think that there are many cases where the numerical values are compared as follows.

x = 10
if x >= 0 and x <= 20:
    print('OK')
else:
    print('NG')

However, in the case of Python, it is possible to write as follows.

x = 10
if 0 <= x <= 20:
    print('OK')
else:
    print('NG')

Ternary operator

When assigning a value to a variable, I think there are many situations where you want to change the value to be assigned depending on the conditions. For example, if the variable is None, or if it is a specific key. At that time, you can write smartly by assigning variables using the ternary operator. The description method of the ternary operator in Python is as follows.

(variable) = (Value when the condition is true) if (Conditional expression) else (Value when the condition is false) 

Code without the ternary operator

test = 15

if test >= 0:
    message = 'Positive number or 0'
else:
    message = 'Negative number'
print(message)

Code when using the ternary operator

test = 15

message = 'Positive number or 0' if test >= 0 else 'Negative number'
print(message)

As mentioned above, since the IF statement part can be described in one line, the assigned part becomes one line, and the readability of the code is improved. Note that in the ternary operator, if you want to branch with multiple conditions (if you want to use elif), you need to nest the ternary operator. However, nesting the ternary operator significantly reduces the readability of the code and is not recommended. If the condition is complicated, the IF statement may be used instead of the ternary operator.

Embed the value of a variable in a string

I think there are four ways to embed variable values in a string in Python.

name = 'taro'

#String+Concatenate with operator
print('I' + name + 'is')

#use f string
print(f'I{name}is')

#use the format function
print('I{}is'.format(name))

# %How to use
print('I%s' % name)

The concatenate strings with the + operator method is not recommended as it makes the code less readable and less editable. (The + operator used for concatenation is meaningless in the first place, and when adding a new one, the + operator must be written.) I wonder if it doesn't matter whether you use the f string or the format function. However, you don't want both descriptions to be used in your project code. I think it should be unified to either one.

Array FOR statement

I think that there are many cases where an array is processed by a FOR statement. Here are some useful techniques to use in that case.

Use FOR EACH

When using the FOR statement normally:

test = [1, 10, 15, 16]

for index in range(len(test)):
    print(test[index], end=' ') # 1 10 15 16 

When using FOR EACH:

test = [1, 10, 15, 16]

for value in test:
    print(value, end=' ') # 1 10 15 16

With FOR EACH, the target to loop in the FOR statement is the element of the array from the index. So if you have a requirement to process the elements of an array one by one, using the FOR EACH notation will improve the readability of your code.

We also often encounter cases where we want to use indexes in FOR EACH. In that case, you can get both the element and the index of the array by doing the following, so you can write smartly.

test = [1, 10, 15, 16]

for index, value in enumerate(test):
    # index: 0 value: 1 , index: 1 value: 10 , index: 2 value: 15 , index: 3 value: 16 , 
    print('index:', index, 'value:', value, ',', end=' ')

Get array elements in reverse order

It's rarely encountered, but it also mentions when you want to get the elements of an array in reverse order.

If you normally get the elements of an array in reverse order:

test = [1, 10, 15, 16]

for i in range(len(test)-1, -1, -1):
    print(test[i], end=' ') # 16 15 10 1 

When using the reversed function:

test = [1, 10, 15, 16]

for value in reversed(test):
    print(value, end=' ') # 16 15 10 1 

In addition to using the reversed function, you can also use slices to reverse the elements of the array. See c-yan's comment for details.

To program smartly in Python, it is important to have some understanding of built-in functions. It describes the range function, len function, reversed function, etc. used in this article. Built-in functions

Dictionary FOR statement

I think that dictionaries are often processed with FOR statements. There are many ways to write a FOR statement in a dictionary, but basically there is no problem if you remember how to use the items function.

When processing a dictionary with a FOR statement normally:

test = {'apple': 'red', 'remon': 'yellow'}

for key in test:
    print('key:', key, 'value:', test[key], ',', end=' ') # key: apple value: red , key: remon value: yellow , 

When using the items function:

test = {'apple': 'red', 'remon': 'yellow'}

for key,value in test.items():
    print('key:', key, 'value:', value, ',', end=' ')

Remove duplicate arrays

I don't encounter it very often, but I think I will encounter it several times. You can check the contents of the array and create your own function to eliminate the duplicates, but you can remove the duplicates more conveniently.

test = [1, 2, 3, 3, 5, 1]
test2 = ['a', 'b', 'a', 'd', 'aa']

print(list(set(test))) # [1, 2, 3, 5]
print(list(set(test2))) # ['a', 'd', 'b', 'aa']

The miso is that you can automatically eliminate duplicates by converting to Set type. After converting to Set type, it is returned to list type again.

Array sort

You don't implement array sorting yourself. It is possible to achieve sorting using existing functions.

test = [10, 25, 4, 100, 69]
print(sorted(test)) # [4, 10, 25, 69, 100]
print(sorted(test, reverse=True)) # [100, 69, 25, 10, 4]

It is also possible to explicitly specify what you want to use as a key in sorting.

test = [[10, 25], [4, 100], [69, 71]]
print(sorted(test, key=lambda item: item[1])) # [[10, 25], [69, 71], [4, 100]]

The code above sorts by the second element in the array as a key.

Sorting the sorted function is not very fast. If you run into speed issues, we recommend using a library such as numpy.

Sort dictionary array

You may want to sort the dictionary array. In that case, you can handle it by explicitly specifying the key as shown below.

test = [{'name': 'taro', 'age': 18},{'name': 'jiro', 'age': 12}]

# [{'name': 'jiro', 'age': 12}, {'name': 'taro', 'age': 18}]
print(sorted(test, key=lambda item: item['age']))

List comprehension

List comprehension is very convenient and it is a description method with fast processing speed, so it is a writing style that you definitely want to learn. List comprehension is described as follows.

[(Value to add to the array) for (One element of an iterator object) in (Iterator object)]

If you normally create an even list:

test = []
for index in range(5):
    test.append(index * 2)
print(test) # [0, 2, 4, 6, 8]

When creating a list in list comprehension:

# [0, 2, 4, 6, 8]
print([index * 2 for index in range(5)])

I was able to write in one line. It is also possible to narrow down the conditions using an IF statement (postfix IF).

#Create an even array that does not contain multiples of 10
# [2, 4, 6, 8, 12, 14, 16, 18, 22, 24, 26, 28, 32, 34, 36, 38]
print([index * 2 for index in range(20) if (index * 2) % 10])

In list comprehension notation, it is also possible to branch the condition with IF and ELSE. In addition, the dictionary also has a comprehension notation, but this time it will be omitted.

all function and any function

Python has an all function that determines if all elements are True and an any function that determines if any element is True. By making good use of the all and any functions, you can write smart programs.

For example, suppose you write a program that determines if the contents of an array are all zeros or greater.

When implementing normally:

test = [-1, 5, 2]
judge = True
for value in test:
    if value < 0:
        judge = False
result = 'The contents are all 0 or more' if judge else 'Contains negative numbers'
print(result)

When using the all function:

test = [-1, 5, 2]
judge = all(map(lambda item: item >= 0, test))
result = 'The contents are all 0 or more' if judge else 'Contains negative numbers'
print(result) #Contains negative numbers

I was able to write 3 lines in 1 line. Now, let's move on to the explanation of the program. First, the map function is a function that allows you to apply a function to each element of an array. When map (lambda item: item> = 0, test) is applied, a new map object is created that determines whether each element is 0 or more. This means that a False, True, True map object will be created. Then, the all function is used to determine whether all the elements are True. One thing to note is that the all function returns an empty array as True, so you need to write a separate branch to see if the array is empty. (Thank you for your comment from marmalade_boy)

Returns True if all elements of iterable are true (or if iterable is empty)

Built-in functions all

Context manager and with block

We won't go into the details of the context manager, but it's a good idea to know some about the with block.

For example, consider opening a file. To handle the situation where an exception occurs while processing the file and the file is not closed, enclose it in a try clause and close the file finally.

fd = open("filename", "r")
try:
    file_content = file.read()
    raise Exception('test')
finally:
   fd.close()

However, it is troublesome to write a try clause every time and close the file, so I use the with block.

with open("filename", "r") as fd:
  file_content = file.read()
  raise Exception('test')

By writing as above, even if an exception occurs in the with block, the file will be closed automatically.

Finally

Thank you for reading the long text. Python is still a deep language, so the ones introduced above are just a few. However, I hope it will be of some help to you. Then (^^ ♪

Recommended Posts

A road to intermediate Python
[Road to Intermediate] Understanding Python Properties
[Road to intermediate Python] Use ternary operators
[Road to intermediate Python] Use lambda expressions
[Road to intermediate Python] Article link summary
[Road to Python Intermediate] Call a class instance like a function with __call__
[Python] Road to a snake charmer (4) Tweak Numpy
[Python] Road to a snake charmer (6) Manipulate Pandas
[Road to Intermediate] Python seems to be all objects
[Road to Python Intermediate] Define __getattr__ function in class
[Road to intermediate Python] Define in in your own class
[Python] Road to a snake charmer (5) Play with Matplotlib
How to write a Python class
[Python] Road to snake charmer (3) Python class
Road to Linux Intermediate: Network Edition
5 Ways to Create a Python Chatbot
[Road to intermediate Python] Install packages in bulk with pip
[Road to intermediate Python] Use if statement in list comprehension
[Road to intermediate Python] Enables comparison operations with original classes
Updated to Python 2.7.9
[Python] How to make a class iterable
[Python] How to convert a 2D list to a 1D list
[Road to Python intermediate] Dynamically specify execution method by variable name
A super introduction to Python bit operations
Send a message from Python to Slack
[Python] Road to snake charmer (1) Environment construction
I want to build a Python environment
[Python] How to invert a character string
How to get a stacktrace in python
A way to understand Python duck typing
Add a Python virtual environment to VSCode
How to run a Maya Python script
The road to compiling to Python 3 with Thrift
"Backport" to python 2
Send a message from Slack to a Python server
Edit Excel from Python to create a PivotTable
How to read a CSV file with Python 2/3
Send a message to LINE with Python (LINE Notify)
Try to calculate a statistical problem in Python
How to create a Python virtual environment (venv)
How to clear tuples in a list (Python)
How to embed a variable in a python string
I want to create a window in Python
Ported a naive homebrew language compiler to Python
How to create a JSON file in Python
Try to draw a life curve with python
I want to make a game with Python
How to generate a Python object from JSON
Try to make a "cryptanalysis" cipher with Python
A clever way to time processing in Python
How to add a Python module search path
Steps to develop a web application in Python
Decide to assign a laboratory with Python (fiction)
[Introduction to Python3 Day 23] Chapter 12 Become a Paisonista (12.1 to 12.6)
Python Note: When assigning a value to a string
Why does Python have to write a colon?
Steps to create a Twitter bot with python
To add a module to python put in Julialang
How to notify a Discord channel in Python
Export Python3 version OpenCV KeyPoint to a file
Try to make a dihedral group with Python