[PYTHON] About the behavior of Queue during parallel processing

Use parallel

test.py


#!/usr/bin/python

import parallel
from Queue           import Queue as Q1
from multiprocessing import Queue as Q2

q1 = Q1()
q2 = Q2()

for i in xrange(6):
    q1.put(i)
    q2.put(i)

def worker(q1, q2):
    print '%6d %6d' % (q1.get(), q2.get())

#--- THREADING
print '\n--- thread ---'
print ' Queue1 Queue2'
print '--------------'
parallel.thread(worker, [q1, q2], 3)

#--- MULTIPROCESS
print '\n--- multiprocess ---'
print ' Queue1 Queue2'
print '--------------------'
parallel.multiprocess(worker, [q1, q2], 3)

result

--- thread ---
 Queue1 Queue2
--------------
     0      0
     1      1
     2      2

--- multiprocess ---
 Queue1 Queue2
--------------------
     3      3
     3      4
     3      5

Conclusion

You should use Queue of multiprocessing

Recommended Posts

About the behavior of Queue during parallel processing
About the behavior of yield_per of SqlAlchemy
Parallel processing with Parallel of scikit-learn
About the behavior of Queue during parallel processing
Parallel processing with multiprocessing
Various processing of Python
Basic processing of librosa
A memo about the behavior of bowtie2 during multiple hits
About the queue
About the behavior of enable_backprop of Chainer v2
About the behavior of copy, deepcopy and numpy.copy
About the processing speed of SVM (SVC) of scikit-learn
About the behavior of Model.get_or_create () of peewee in Python
About the ease of Python
Parallel processing with Parallel of scikit-learn
About the features of Python
Tank game made with python About the behavior of tanks
About the return value of pthread_mutex_init ()
About the basic type of Go
About the upper limit of threads-max
About the size of matplotlib points
About color halftone processing of images
About the basics list of Python basics
Restart the server during Fabric processing
Check the behavior of destructor in Python
Let's talk about the tone curve of image processing ~ LUT is amazing ~
About the virtual environment of python version 3.7
About the arguments of the setup function of PyCaret
Receive a list of the results of parallel processing in Python with starmap
About the Normal Equation of Linear Regression
The story of blackjack A processing (python)
About the main tasks of image processing (computer vision) and the architecture used
About the accuracy of Archimedean circle calculation method
See the behavior of drunkenness with reinforcement learning
View the result of geometry processing in Python
The behavior of signal () depends on the compile options
A note about the python version of python virtualenv
Image processing? The story of starting Python for
I checked the processing speed of numpy one-dimensionalization
About the development contents of machine learning (Example)
[Note] About the role of underscore "_" in Python
Visualize the behavior of the sorting algorithm with matplotlib
About the * (asterisk) argument of python (and itertools.starmap)
Behavior of multiprocessing.pool.Pool.map
A memorandum about the warning of the pylint output result
Take a peek at the processing of LightGBM Tuner
Think about the next generation of Rack and WSGI
About testing in the implementation of machine learning models
About the inefficiency of data transfer in luigi on-memory
Examine the close processing of Python dataset (SQLAlchemy wrapper)
About the uncluttered arrangement in the import order of flake8
A story about changing the master name of BlueZ
Personal notes about the integration of vscode and anaconda
A reminder about the implementation of recommendations in Python
Confirmed the difference in the presence or absence of random processing during mini-batch learning with chainer