How can I tell if I am overusing multi-threading?
Posted
by
exhuma
on Programmers
See other posts from Programmers
or by exhuma
Published on 2013-11-06T13:25:56Z
Indexed on
2013/11/06
16:10 UTC
Read the original article
Hit count: 202
python
|multithreading
NOTE: This is a complete re-write of the question. The text before was way too lengthy and did not get to the point! If you're interested in the original question, you can look it up in the edit history.
I currently feel like I am over-using multi-threading.
I have 3 types of data, A, B and C.
Each A
can be converted to multiple B
s and each B
can be converted to multiple C
s.
I am only interested in treating C
s.
I could write this fairly easily with a couple of conversion functions. But I caught myself implementing it with threads, three queues (queue_a
, queue_b
and queue_c
). There are two threads doing the different conversions, and one worker:
ConverterA
reads fromqueue_a
and writes toqueue_b
ConverterB
reads fromqueue_b
and writes toqueue_c
Worker
handles each element fromqueue_c
The conversions are fairly mundane, and I don't know if this model is too convoluted. But it seems extremely robust to me. Each "converter" can start working even before data has arrived on the queues, and at any time in the code I can just "submit" new A
s or B
s and it will trigger the conversion pipeline which in turn will trigger a job by the worker thread.
Even the resulting code looks simpler. But I still am unsure if I am abusing threads for something simple.
© Programmers or respective owner