Python Multiprocessing with Queue vs ZeroMQ IPC
- by Imraan
I am busy writing a Python application using ZeroMQ and implementing a variation of the Majordomo pattern as described in the ZGuide.
I have a broker as an intermediary between a set of workers and clients. I want to do some extensive logging for every request that comes in, but I do not want the broker to waste time doing that. The broker should pass that logging request to something else.
I have thought of two ways :-
Create workers that are only for logging and use the ZeroMQ IPC transport
Use Multiprocessing with a Queue
I am not sure which one is better or faster for that matter. The first option does allow me to use the current worker base classes that I already use for normal workers, but the second option seems quicker to implement.
I would like some advice or comments on the above or possibly a different solution.