When performing parallel processing with Python, I was confused about how to write in the following cases, so I summarized it as a memorandum.

- Use multiprocessing.Pool
- Use map method
- The called function has multiple return values
- I want to receive these return values and do some processing

- OS - macOS Catalina 10.15.7
- CPU - Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
- RAM - 16 GB 2133 MHz LPDDR3
- Python 3.8.6

This time, I made the following sample code as an example.

- If you pass two values, prepare a function that finds the greatest common divisor and the least common multiple and returns each.
- Try calling each with and without parallel

This simply calls the function for the length of the list `NUMBERS`

.

`python`

```
import time
def calc(num_pair):
x, y = num_pair
low = min(x, y)
gcd = 0
# Greatest common divisor
for i in range(low, 0, -1):
if x % i == 0 and y % i == 0:
gcd = i
break
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
gcd = []
lcm = []
for i, pair in enumerate(NUMBERS):
g, l = calc(NUMBERS[i])
gcd.append(g)
lcm.append(l)
print(f'gcd = {gcd}')
# gcd = [2, 10, 1, 3, 1, 2, 1, 1]
print(f'lcm = {lcm}')
# lcm = [152347185038394, 329630179524830, 6207207196267247, 672939406483941, 4719653803780070, 440287465538522, 4039567667958797, 8803120073065635]
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
#The output result is "with parallel"
end = time.time()
print(f'Time = {(end-start):.3f}')
#Result of measuring 3 times
# Time = 29.068
# Time = 29.796
# Time = 29.890
```

The point is the content of the result received by `p.map ()`

.
It's hard to know that the more the return value of a function, the more it fits into one tuple of result, but if you don't know it, you'll want to receive the value as written without parallel.

```
from multiprocessing import Pool
import time
def calc(num_pair):
x, y = num_pair
low = min(x, y)
gcd = 0
# Greatest common divisor
for i in range(low, 0, -1):
if x % i == 0 and y % i == 0:
gcd = i
break
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
p = Pool(8)
try:
result = p.map(calc, NUMBERS)
except Exception as e:
print(e)
print(f'result = {result}')
# result = [(2, 152347185038394), (10, 329630179524830), (1, 6207207196267247), (3, 672939406483941), (1, 4719653803780070), (2, 440287465538522), (1, 4039567667958797), (1, 8803120073065635)]
gcd = [i[0] for i in result]
lcm = [j[1] for j in result]
print(f'gcd = {gcd}')
# gcd = [2, 10, 1, 3, 1, 2, 1, 1]
print(f'lcm = {lcm}')
# lcm = [152347185038394, 329630179524830, 6207207196267247, 672939406483941, 4719653803780070, 440287465538522, 4039567667958797, 8803120073065635]
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
#The greatest common divisor of 12345678 and 24680246 is 2.,The least common multiple is 152347185038394
#The greatest common divisor of 91845010 and 35889830 is 10.,The least common multiple is 329630179524830
#The greatest common divisor of 82163657 and 75546871 is 1.,The least common multiple is 6207207196267247
#The greatest common divisor of 46015383 and 43872681 is 3,The least common multiple is 672939406483941
#The greatest common divisor of 73739990 and 64003993 is 1.,The least common multiple is 4719653803780070
#The greatest common divisor of 26514146 and 3321514 is 2,The least common multiple is 440287465538522
#The greatest common divisor of 51395783 and 78597259 is 1.,The least common multiple is 4039567667958797
#The greatest common divisor of 99939535 and 88084461 is 1.,The least common multiple is 8803120073065635
end = time.time()
print(f'Time = {(end-start):.3f}')
#Result of measuring 3 times
# Time = 18.861
# Time = 16.983
# Time = 18.362
```

If you call a function that returns multiple values in Pool and map, it will be returned as a list of tuples stored for the number of returned values.

This time, I made the above code intentionally to make it a time-consuming process for explanation, but if you want to find the greatest common divisor really quickly, it is better to use the existing library as follows.

```
import math
from multiprocessing import Pool
import time
def calc(num_pair):
x, y = num_pair
# Greatest common divisor
gcd = math.gcd(x,y)
# Least common multiple
lcm = x * y // gcd
return gcd, lcm
if __name__ == "__main__":
result=[]
NUMBERS = [
(12345678,24680246),(91845010,35889830),
(82163657,75546871),(46015383,43872681),
(73739990,64003993),(26514146,33211514),
(51395783,78597259),(99939535,88084461)
]
start = time.time()
gcd = []
lcm = []
for i, pair in enumerate(NUMBERS):
g, l = calc(NUMBERS[i])
gcd.append(g)
lcm.append(l)
for i, pair in enumerate(NUMBERS):
print(f'{pair[0]}When{pair[1]}The greatest common divisor of{gcd[i]},The least common multiple is{lcm[i]}is')
end = time.time()
print(f'Time = {(end-start):.5f}')
# Time = 0.00013
```

It's much faster to use `math.gcd ()`

.
When parallel processing is used here, for some reason `Time = 0.17145`

and no parallel processing finished faster.

Parallel processing is also quite profound.

Recommended Posts