[Introduction to Python3 Day 23] Chapter 12 Become a Paisonista (12.1 to 12.6)

12.1 Code testing

12.1.1 Check by pylint, pyflakes, pep8

style1



a=1
b=2
print(a)
print(b)
print(c)

Execution result



$ pylint style1.py
************* Module style1
style1.py:1:1: C0326: Exactly one space required around assignment
a=1
 ^ (bad-whitespace)
style1.py:2:1: C0326: Exactly one space required around assignment
b=2
 ^ (bad-whitespace)
style1.py:1:0: C0114: Missing module docstring (missing-module-docstring)
style1.py:1:0: C0103: Constant name "a" doesn't conform to UPPER_CASE naming style (invalid-name)
style1.py:2:0: C0103: Constant name "b" doesn't conform to UPPER_CASE naming style (invalid-name)
#An E at the beginning indicates an error.
style1.py:5:6: E0602: Undefined variable 'c' (undefined-variable)

-------------------------------------
Your code has been rated at -10.00/10

Add the value of c to resolve the error.

style2.py



a=1
b=2
c=3
print(a)
print(b)
print(c)

Execution result


$ pylint style2.py
************* Module style2
style2.py:1:1: C0326: Exactly one space required around assignment
a=1
 ^ (bad-whitespace)
style2.py:2:1: C0326: Exactly one space required around assignment
b=2
 ^ (bad-whitespace)
style2.py:3:1: C0326: Exactly one space required around assignment
c=3
 ^ (bad-whitespace)
style2.py:1:0: C0114: Missing module docstring (missing-module-docstring)
style2.py:1:0: C0103: Constant name "a" doesn't conform to UPPER_CASE naming style (invalid-name)
style2.py:2:0: C0103: Constant name "b" doesn't conform to UPPER_CASE naming style (invalid-name)
style2.py:3:0: C0103: Constant name "c" doesn't conform to UPPER_CASE naming style (invalid-name)

------------------------------------
Your code has been rated at -1.67/10

Try lengthening the variable name.

style3.py


def func():

    first=1
    second=2
    third=3
    print(first)
    print(second)
    print(third)

func()

Execution result


$ pylist style3.py
-bash: pylist: command not found
uemuratntonoAir:bin uemura$ pylint style3.py
************* Module style3
style3.py:3:9: C0326: Exactly one space required around assignment
    first=1
         ^ (bad-whitespace)
style3.py:4:10: C0326: Exactly one space required around assignment
    second=2
          ^ (bad-whitespace)
style3.py:5:9: C0326: Exactly one space required around assignment
    third=3
         ^ (bad-whitespace)
style3.py:1:0: C0114: Missing module docstring (missing-module-docstring)
style3.py:1:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 3.75/10

12.1.2 Test by unit test

--Before committing your code to the source control system, it's a good practice to write a test program first and make sure your code passes all the tests.

cap.py



def just_do_it(text):
     return text.capitalize()

test_cap.py


import unittest
import cap

class TestCap(unittest.TestCase):

    def setUp(self):
        pass

    def tearDown(self):
        pass

    def test_one_word(self):
        text="duck"
        result=cap.just_do_it(text)
        self.assertEqual(result, "Duck")

    def test_multiple_words(self):
        text="a veritable flock of ducks"
        result=cap.just_do_it(text)
        self.assertEqual(result, "A Veritable Flock Of Ducks")

#Check the result with a method whose name starts with assert.

if __name__=="__main__":
    unittest.main()

Execution result



$ python test_cap.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK
$ python test_cap.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK
$ python test_cap.py
..F
======================================================================
FAIL: test_words_with_apostrophes (__main__.TestCap)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_cap.py", line 25, in test_words_with_apostrophes
    self.assertEqual(result, "I'm Fresh Out Of Ideas")
AssertionError: "I'M Fresh Out Of Ideas" != "I'm Fresh Out Of Ideas"

#^Indicates where the strings are actually different.
- I'M Fresh Out Of Ideas
?   ^
+ I'm Fresh Out Of Ideas
?   ^


----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

capitalize () makes only the first word a title case.

cap.py


#capitalize()Title()Rewrite to
def just_do_it(text):
     return text.title()

Execution result



$ python test_cap.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

test_cap.py



import unittest
import cap

class TestCap(unittest.TestCase):

    def setUp(self):
        pass

    def tearDown(self):
        pass

    def test_one_word(self):
        text="duck"
        result=cap.just_do_it(text)
        self.assertEqual(result, "Duck")

    def test_multiple_words(self):
        text="a veritable flock of ducks"
        result=cap.just_do_it(text)
        self.assertEqual(result, "A Veritable Flock Of Ducks")

#Check the result with a method whose name starts with assert.

    def test_words_with_apostrophes(self):
        text="I'm fresh out of ideas"
        result=cap.just_do_it(text)
        self.assertEqual(result, "I'm Fresh Out Of Ideas")

    def test_words_with_quotes(self):
        text="\"You're despicable,\" said Daffy Duck"
        result=cap.just_do_it(text)
        self.assertEqual(result, "\"You're Despicable,\" Said Daffy Duck")


if __name__=="__main__":
    unittest.main()

Execution result



$ python test_cap.py
...F
======================================================================
FAIL: test_words_with_quotes (__main__.TestCap)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_cap.py", line 30, in test_words_with_quotes
    self.assertEqual(result, "\"You're Despicable,\" Said Daffy Duck")
AssertionError: '"you\'re Despicable," Said Daffy Duck' != '"You\'re Despicable," Said Daffy Duck'
- "you're Despicable," Said Daffy Duck
?  ^
+ "You're Despicable," Said Daffy Duck
?  ^


----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=1)


12.1.3 Nose test

--nose is a third party package.

test_cap_nose.py



import cap
from nose.tools import eq_

    def test_one_word(self):
        text="duck"
        result=cap.just_do_it(text)
        eq_(result, "Duck")

    def test_multiple_words(self):
        text="a veritable flock of ducks"
        result=cap.just_do_it(text)
        eq_(result, "A Veritable Flock Of Ducks")

    def test_words_with_apostrophes(self):
        text="I'm fresh out of ideas"
        result=cap.just_do_it(text)
        eq_(result, "I'm Fresh Out Of Ideas")

    def test_words_with_quotes(self):
        text="\"You're despicable,\" said Daffy Duck"
        result=cap.just_do_it(text)
        eq_(result, "\"You're Despicable,\" Said Daffy Duck")


The same bug I found when testing with unittestn.

Execution result



$ nosetests test_cap_nose.py
E
======================================================================
ERROR: Failure: IndentationError (unexpected indent (test_cap_nose.py, line 4))
----------------------------------------------------------------------
...(abridgement)
    def test_one_word(self):
    ^
IndentationError: unexpected indent

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

12.2 Debugging Python code

test_dump.py



from dump1 import dump

@dump
def double(*args, **kwargs):
    "Double every argument"
    output_list=[2*x for x in args]
    output_dict={k:2*v for k,v in kwargs.items()}
    return output_list, output_dict

if __name__=="__main__":
    output=double(3, 5, first=100,next=98.6, last=-40)

Execution result


$ python test_dump.py
Function name: double
Input arguments: 3 5
Input keyword arguments: dict_items([('first', 100), ('next', 98.6), ('last', -40)])
Output: ([6, 10], {'first': 200, 'next': 197.2, 'last': -80})


12.3 Debugging with pdb

capitals.py


def process_cities(filename):
    with open(filename, "rt") as file:
        for line in file:
            line=line.strip()
            if "quit" in line.lower():
                return
            country, city=line.split(",")
            city=city.strip()
            country=country.strip()
            print(city.title(),country.title(), sep=",")

if __name__=="__main__":
    import sys
    process_cities(sys.argv[1])

Execution result


$ python capitals.py cities1.csv
Paris,France
Caracas,Venuzuela
Vilnius,Lithunia


While executing cities2.csv, it stops when only 5 out of 15 lines are output.

cities2.csv


$ python capitals.py cities2.csv
Buenos Aires,Argentinam
La Paz,Bolivia
Brasilia,Brazil
Santiago,Chile
Bogota,Colombia


Execution result



$ python -m pdb capitals.py cities2.csv

#If you enter c, it ends normally or stops with an error.
(Pdb) c
Buenos Aires,Argentinam
La Paz,Bolivia
Brasilia,Brazil
Santiago,Chile
Bogota,Colombia
The program finished and will be restarted
> /Users/practice/bin/capitals.py(1)<module>()
-> def process_cities(filename):

#Type s to execute the lines in sequence.
(Pdb) s
> /Users/practice/bin/capitals.py(13)<module>()
-> import sys
(Pdb) s
> /Users/practice/bin/capitals.py(14)<module>()
-> process_cities(sys.argv[1])
(Pdb) s
--Call--
> /Users/practice/bin/capitals.py(1)process_cities()
-> def process_cities(filename):
(Pdb) s
> /Users/practice/bin/capitals.py(2)process_cities()
-> with open(filename, "rt") as file:

#I set a breakpoint on the 6th line.
(Pdb) b 6
Breakpoint 1 at /Users/uemura/practice/bin/capitals.py:6

#Execute the program until it hits a breakpoint or reads all the input lines and finishes.
(Pdb) c
Buenos Aires,Argentinam
La Paz,Bolivia
Brasilia,Brazil
Santiago,Chile
Bogota,Colombia
> /Users/practice/bin/capitals.py(6)process_cities()
-> return
#Check what you read.
(Pdb) p line
'ecuador,quito'
(Pdb) b
Num Type         Disp Enb   Where
1   breakpoint   keep yes   at /Users/uemura/practice/bin/capitals.py:6
	breakpoint already hit 1 time
#1 is the line of code, current position(->),breakpoint(B)Is shown.
#Since l without an option starts displaying from the end of the previous l display, specify it as an option.
(Pdb) l 1
  1  	def process_cities(filename):
  2  	    with open(filename, "rt") as file:
  3  	        for line in file:
  4  	            line=line.strip()
  5  	            if "quit" in line.lower():
  6 B->	                return
  7  	            country, city=line.split(",")
  8  	            city=city.strip()
  9  	            country=country.strip()
 10  	            print(city.title(),country.title(), sep=",")
 11  	


Rewrite the quit test to match only those that are quit on the entire line.

capitals2.py


def process_cities(filename):
    with open(filename, "rt") as file:
        for line in file:
            line=line.strip()
            if "quit" ==line.lower():
                return
            country, city=line.split(",")
            city=city.strip()
            country=country.strip()
            print(city.title(),country.title(), sep=",")

if __name__=="__main__":
    import sys
    process_cities(sys.argv[1])

Execution result



$ python capitals2.py cities2.csv
Buenos Aires,Argentinam
La Paz,Bolivia
Brasilia,Brazil
Santiago,Chile
Bogota,Colombia
Quito,Ecuador
Stanley,Falkland Islands
Cayenne,French Guiana
Asuncion,Paraguay
Lima,Peru
Paramaribo,Suriname
Montevideo,Uruguay
Caracas,Venezuela

12.4 Error message logging

--The Python standard library for logging is the logging module. --Messages you want to save in the log --Priority level for ranking and corresponding function --One or more logger objects that are the main communication path with the module --Handlers that send messages to terminals, files, etc. --Formatter that creates output --Filters that make decisions based on input

>>> import logging
>>> logging.debug("Looks like rain")
>>> logging.info("And hill")
#The default priority level is WARNIG.
>>> logging.warn("Did I hear thunder?")
__main__:1: DeprecationWarning: The 'warn' function is deprecated, use 'warning' instead
WARNING:root:Did I hear thunder?
>>> logging.critical("Stop fencing and get inside")
CRITICAL:root:Stop fencing and get inside

#The default level is basicConfig()Can be set with.
#All levels above DEBUG are logged.
>>> import logging
>>> logging.basicConfig(level="DEBUG")
>>> logger.debug("Timber")
DEBUG:bunyan:Timber
>>> logging.info("And hill")
INFO:root:And hill

12.5 Code optimization

12.5.1 Measurement of execution time

time1.py


#Subtract the processing end time from the current time.
from time import time

t1=time()
num=5
num*=2
print(time()-t1)

$ python time1.py
3.0994415283203125e-06
$ python time1.py
1.9073486328125e-06
$ python time1.py
1.6689300537109375e-06


time2.py



from time import time, sleep

t1=time()
sleep(1.0)
print(time()-t1)

Execution result



$ python time2.py
1.0051441192626953
uemuratntonoAir:bin uemura$ python time2.py
1.0001447200775146

It is convenient to use the timeit () function of the timeit module.

timeit1.py


#time.timeit("code",number,count)Syntax
#Since it is executed inside timeit, the code is""Must be enclosed in.

from timeit import timeit
print(timeit("num=5;num*=2", number=1))


Execution result



$ python timeit1.py
1.63200000000141e-06
uemuratntonoAir:bin uemura$ python timeit1.py
1.252999999999671e-06


time2.py


#repeat()You can increase the number of executions by using the repeat argument of the function.
from timeit import repeat
print(repeat("num=5;num*=2", number=1,repeat=3))


Execution result


$ python timeit2.py
[8.809999999977169e-07, 2.57000000000035e-07, 1.659999999993611e-07]


12.5.2 Algorithm and data structure

Which is faster, list comprehension or for loop?

time_lists.py



from timeit import timeit

def make1():
     result=[]
     for value in range(1000):
         result.append(value)
     return result

def make2():
    result=[value for value in range(1000)]
    return result

print("make1 takes", timeit(make1, number=1000), "seconds")
print("make2 takes", timeit(make2, number=1000), "seconds")


List comprehensions are faster.

Execution result



$ python time_lists.py
make1 takes 0.07954489899999999 seconds
make2 takes 0.035908797000000006 seconds

12.6 Source control with Git


#Move after creating a new directory.
$ mkdir newdir
$ cd newdir

#Create a local Git repository in newdir in your current directory.
$ git init
Initialized empty Git repository in /Users/practice/bin/newdir/.git/

#Add files to your Git repository.
$ git add test.py

#Check the status of Git.
$ git status
On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

	new file:   test.py

#-m "my first commit"Commit with the commit message.
$ git commit -m "my first commit"
[master (root-commit) e03cd1c] my first commit
 1 file changed, 1 insertion(+)
 create mode 100644 test.py

#Changes have been committed.
$ git status
On branch master
nothing to commit, working tree clean


Impressions

It was the first lap, but I was able to finish the race in about a month. However, since the output has not been produced, I will focus on the output for the next month.

References

"Introduction to Python3 by Bill Lubanovic (published by O'Reilly Japan)"

Recommended Posts

[Introduction to Python3 Day 23] Chapter 12 Become a Paisonista (12.1 to 12.6)
[Introduction to Python3 Day 14] Chapter 7 Strings (7.1.1.1 to 7.1.1.4)
[Introduction to Python3 Day 15] Chapter 7 Strings (7.1.2-7.1.2.2)
[Introduction to Python3 Day 21] Chapter 10 System (10.1 to 10.5)
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.1-8.2.5)
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.3-8.3.6.1)
[Introduction to Python3 Day 19] Chapter 8 Data Destinations (8.4-8.5)
[Introduction to Python3 Day 18] Chapter 8 Data Destinations (8.3.6.2 to 8.3.6.3)
[Introduction to Python3 Day 12] Chapter 6 Objects and Classes (6.3-6.15)
[Introduction to Python3 Day 22] Chapter 11 Concurrency and Networking (11.1 to 11.3)
[Introduction to Python3 Day 11] Chapter 6 Objects and Classes (6.1-6.2)
[Introduction to Python3 Day 20] Chapter 9 Unraveling the Web (9.1-9.4)
[Introduction to Python3 Day 8] Chapter 4 Py Skin: Code Structure (4.1-4.13)
[Introduction to Python3 Day 1] Programming and Python
[Introduction to Python3 Day 3] Chapter 2 Py components: Numbers, strings, variables (2.2-2.3.6)
[Introduction to Python3 Day 2] Chapter 2 Py Components: Numbers, Strings, Variables (2.1)
[Introduction to Python3 Day 4] Chapter 2 Py Components: Numbers, Strings, Variables (2.3.7-2.4)
A super introduction to Python bit operations
Introduction to Effectiveness Verification Chapter 1 in Python
[Introduction to Python3 Day 5] Chapter 3 Py Tools: Lists, Tuples, Dictionaries, Sets (3.1-3.2.6)
[Introduction to Python3 Day 10] Chapter 5 Py's Cosmetic Box: Modules, Packages, Programs (5.4-5.7)
[Introduction to Python3 Day 9] Chapter 5 Py's Cosmetic Box: Modules, Packages, Programs (5.1-5.4)
[Introduction to Python3 Day 6] Chapter 3 Py tool lists, tuples, dictionaries, sets (3.2.7-3.2.19)
Introduction to effectiveness verification Chapter 3 written in Python
Introduction to Python language
Introduction to OpenCV (python)-(2)
Introduction to Effectiveness Verification Chapter 2 Written in Python
[Chapter 5] Introduction to Python with 100 knocks of language processing
[Introduction to python] A high-speed introduction to Python for busy C ++ programmers
[Chapter 3] Introduction to Python with 100 knocks of language processing
[Chapter 2] Introduction to Python with 100 knocks of language processing
Introduction to Linear Algebra in Python: A = LU Decomposition
[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-
[Chapter 4] Introduction to Python with 100 knocks of language processing
Introduction to Python Django (2) Win
A road to intermediate Python
A super introduction to Linux
Introduction to serial communication [Python]
[Introduction to Python] <list> [edit: 2020/02/22]
Introduction to Python (Python version APG4b)
An introduction to Python Programming
Introduction to Python For, While
I can't sleep until I build a server !! (Introduction to Python server made in one day)
Day 68 [Introduction to Kaggle] Random Forest was a simple one.
[Introduction to Udemy Python 3 + Application] 58. Lambda
[Introduction to Udemy Python 3 + Application] 31. Comments
How to write a Python class
Introduction to Python Numerical Library NumPy
Practice! !! Introduction to Python (Type Hints)
[Introduction to Python] <numpy ndarray> [edit: 2020/02/22]
Introduction to Python Hands On Part 1
[Introduction to Python] How to parse JSON
[Introduction to Udemy Python 3 + Application] 56. Closure
Introduction to Protobuf-c (C language ⇔ Python)
[Introduction to Udemy Python3 + Application] 59. Generator
[Introduction to Python] Let's use pandas
A light introduction to object detection
[Introduction to Python] Let's use pandas
5 Ways to Create a Python Chatbot
[Introduction to Udemy Python 3 + Application] Summary
Introduction to image analysis opencv python