There is a way to use it late in any language. I've used the method of using it late It's easy to say, "The language XX, the library is slow." Here are some things that beginners tend to do.
-In order to add a list item, the result of concatenating to the list with the + operator is assigned to the list.
ex.py
a = []
for i in range(1000):
a = a + [i]
-Despite the fact that there is an effective method to add the list to the list It repeats appending individual elements. (In the case of python, there is an extend method.)
・ When the data structure is not properly used according to the purpose Lists are used for sufficient processing in dictionaries and sets. It is O (N) to see if the list contains values if x in listData: Whether it is included in the dictionary key is O (1) if dictData.has_key(x): Whether it is included in the set set is O (1) if x in setData:
・ Without noticing that it can be executed effectively by using the functions included in the standard library. I make a slow implementation with my own library.
In a library that has a data format called a set, find the union, intersection, and difference of the set. Methods are provided. For Python set type s.union(t) s.intersection(t) s.difference(t)
With them, you'll most likely end up with a concise program that runs faster than your own library. I will.
-Make a copy of an unnecessary object. Example: A language / library that allows subarrays to be described concisely and passed as arguments without making a copy. Despite this, I make a copy and hand it over. In C ++, the OpenCV cv :: Mat type can pass a partial image as a function argument. Python's Numpy array type can also pass a subarray as a function argument.
Example: In C ++, you can pass by reference, but you pass by value, Make a copy of an unnecessary object. (With an inadequate text curriculum, C ++ is sufficient to teach only passing by value and passing pointers.)
・ Despite the fact that the library is designed to perform matrix calculations effectively In such a library, matrix operations are processed in a loop of for statements. (MATLAB, python numpy, etc.)
ex2.py
# -*- coding: utf-8 -*-
def func1():
b=['d', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's']
for i in range(10000):
a=['a', 'b', 'c']
for x in b:
a= a+[x]
# print a
def func2():
b=['d', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's']
for i in range(10000):
a=['a', 'b', 'c']
for x in b:
a.append(x)
# print a
def func3():
b=['d', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's']
for i in range(10000):
a=['a', 'b', 'c']
a = a+b
# print a
def func4():
b=['d', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's']
for i in range(10000):
a=['a', 'b', 'c']
a.extend(b)
# print a
if __name__=='__main__':
func1()
func2()
func3()
func4()
In the case of Python, you can take a profile without rewriting the script by using [Run] [Profile] in the Spyder integrated environment.
Summary: Use append and extend methods to add data to the list. It is not a good idea to repeat the assignment using the + operator. Use different data structures such as lists, dictionaries, and sets. Use the data structure methods found in standard libraries and avoid creating your own equivalent functionality. Do not make unnecessary copies of your data. Do not use for loops that can be executed without for loops in array operations.
Postscript: If you want to know more about these contents, "High Performance Python" You should read. At the time of writing this article, this Japanese translation had not been published.
Reference information Speed comparison for adding Python lists (append, comprehension, etc.) http://nonbiri-tereka.hatenablog.com/entry/2014/10/20/110304
Recommended Posts