When performing parallel processing with Python, I was confused about how to write in the following cases, so I summarized it as a memorandum.
This time, I made the following sample code as an example.
This simply calls the function for the length of the list NUMBERS.
python
import time
def calc(num_pair):
x, y = num_pair
low = min(x, y)
gcd = 0
# Greatest common divisor
for i in range(low, 0, -1):
if x % i == 0 and y % i == 0:
gcd = i
break
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
gcd = []
lcm = []
for i, pair in enumerate(NUMBERS):
g, l = calc(NUMBERS[i])
gcd.append(g)
lcm.append(l)
print(f'gcd = {gcd}')
# gcd = [2, 10, 1, 3, 1, 2, 1, 1]
print(f'lcm = {lcm}')
# lcm = [152347185038394, 329630179524830, 6207207196267247, 672939406483941, 4719653803780070, 440287465538522, 4039567667958797, 8803120073065635]
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
#The output result is "with parallel"
end = time.time()
print(f'Time = {(end-start):.3f}')
#Result of measuring 3 times
# Time = 29.068
# Time = 29.796
# Time = 29.890
The point is the content of the result received by p.map ().
It's hard to know that the more the return value of a function, the more it fits into one tuple of result, but if you don't know it, you'll want to receive the value as written without parallel.
from multiprocessing import Pool
import time
def calc(num_pair):
x, y = num_pair
low = min(x, y)
gcd = 0
# Greatest common divisor
for i in range(low, 0, -1):
if x % i == 0 and y % i == 0:
gcd = i
break
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
p = Pool(8)
try:
result = p.map(calc, NUMBERS)
except Exception as e:
print(e)
print(f'result = {result}')
# result = [(2, 152347185038394), (10, 329630179524830), (1, 6207207196267247), (3, 672939406483941), (1, 4719653803780070), (2, 440287465538522), (1, 4039567667958797), (1, 8803120073065635)]
gcd = [i[0] for i in result]
lcm = [j[1] for j in result]
print(f'gcd = {gcd}')
# gcd = [2, 10, 1, 3, 1, 2, 1, 1]
print(f'lcm = {lcm}')
# lcm = [152347185038394, 329630179524830, 6207207196267247, 672939406483941, 4719653803780070, 440287465538522, 4039567667958797, 8803120073065635]
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
#The greatest common divisor of 12345678 and 24680246 is 2.,The least common multiple is 152347185038394
#The greatest common divisor of 91845010 and 35889830 is 10.,The least common multiple is 329630179524830
#The greatest common divisor of 82163657 and 75546871 is 1.,The least common multiple is 6207207196267247
#The greatest common divisor of 46015383 and 43872681 is 3,The least common multiple is 672939406483941
#The greatest common divisor of 73739990 and 64003993 is 1.,The least common multiple is 4719653803780070
#The greatest common divisor of 26514146 and 3321514 is 2,The least common multiple is 440287465538522
#The greatest common divisor of 51395783 and 78597259 is 1.,The least common multiple is 4039567667958797
#The greatest common divisor of 99939535 and 88084461 is 1.,The least common multiple is 8803120073065635
end = time.time()
print(f'Time = {(end-start):.3f}')
#Result of measuring 3 times
# Time = 18.861
# Time = 16.983
# Time = 18.362
If you call a function that returns multiple values in Pool and map, it will be returned as a list of tuples stored for the number of returned values.
This time, I made the above code intentionally to make it a time-consuming process for explanation, but if you want to find the greatest common divisor really quickly, it is better to use the existing library as follows.
import math
from multiprocessing import Pool
import time
def calc(num_pair):
x, y = num_pair
# Greatest common divisor
gcd = math.gcd(x,y)
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
gcd = []
lcm = []
for i, pair in enumerate(NUMBERS):
g, l = calc(NUMBERS[i])
gcd.append(g)
lcm.append(l)
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
end = time.time()
print(f'Time = {(end-start):.5f}')
# Time = 0.00013
It's much faster to use math.gcd ().
When parallel processing is used here, for some reason Time = 0.17145 and no parallel processing finished faster.
Parallel processing is also quite profound.
Recommended Posts