[Python] Passing values during multiprocessing

Leave a note as the value could not be passed when passing variables between Python multiprocesses

Implementation

Define four functions (f1, f2, f3, f4) and execute each function in a separate process. Each function appends a value and confirms the value added from the last main process.

NG

Since the testlist defined to store variables is a memory space that exists only in the main process, it seems that reference update is not possible from another process.

--Program

from multiprocessing import Process
from multiprocessing import Manager

testlist=[]
def f1():
    #print("function f1")
    testlist.append("f1")

def f2():
    #print("function f2")
    testlist.append("f2")

def f3():
    #print("function f3")
    testlist.append("f3")

def f4():
    #print("function f4")
    testlist.append("f4")

if __name__ == '__main__':
    for i in (0,1,2):
        p1 = Process(target=f1, args=())
        p1.start()
        p2 = Process(target=f2, args=())
        p2.start()
        p3 = Process(target=f3, args=())
        p3.start()
        p4 = Process(target=f4, args=())
        p4.start()

        p1.join()
        p2.join()
        p3.join()
        p4.join()
        p1.terminate()
        p2.terminate()
        p3.terminate()
        p4.terminate()
    print("")
    print("result: ",testlist)

--Result

result:  []

OK

Use Manager to pass variables. Set list as an argument and pass data between processes. This makes it possible to update the value from each function and check it from the main process.

--Program

from multiprocessing import Process
from multiprocessing import Manager

def f1(dummy,testlist,num):
    #print("function f1")
    testlist.append("f1")

def f2(dummy,testlist,num):
    #print("function f2")
    testlist.append("f2")

def f3(dummy,testlist,num):
    #print("function f3")
    testlist.append("f3")

def f4(dummy,testlist,num):
    #print("function f4")
    testlist.append("f4")


if __name__ == '__main__':

    alllist=[]
    with Manager() as manager:
        for i in (0,1,2):
            d=manager.dict()
            l=manager.list([])
            p1 = Process(target=f1, args=(d,l,4*i+0))
            p1.start()
            p2 = Process(target=f2, args=(d,l,4*i+1))
            p2.start()
            p3 = Process(target=f3, args=(d,l,4*i+2))
            p3.start()
            p4 = Process(target=f4, args=(d,l,4*i+3))
            p4.start()


            p1.join()
            p2.join()
            p3.join()
            p4.join()
            p1.terminate()
            p2.terminate()
            p3.terminate()
            p4.terminate()
            alllist.extend(l)
    print("")
    print("result: ",alllist)

--Result

result:  ['f1', 'f2', 'f3', 'f4', 'f1', 'f2', 'f3', 'f4', 'f1', 'f2', 'f3', 'f4']