How can I tell if I am overusing multi-threading?

Posted by exhuma on Programmers See other posts from Programmers or by exhuma
Published on 2013-11-06T13:25:56Z Indexed on 2013/11/06 16:10 UTC
Read the original article Hit count: 202

Filed under:
|

NOTE: This is a complete re-write of the question. The text before was way too lengthy and did not get to the point! If you're interested in the original question, you can look it up in the edit history.


I currently feel like I am over-using multi-threading.

I have 3 types of data, A, B and C.

Each A can be converted to multiple Bs and each B can be converted to multiple Cs.

I am only interested in treating Cs.

I could write this fairly easily with a couple of conversion functions. But I caught myself implementing it with threads, three queues (queue_a, queue_b and queue_c). There are two threads doing the different conversions, and one worker:

  • ConverterA reads from queue_a and writes to queue_b
  • ConverterB reads from queue_b and writes to queue_c
  • Worker handles each element from queue_c

The conversions are fairly mundane, and I don't know if this model is too convoluted. But it seems extremely robust to me. Each "converter" can start working even before data has arrived on the queues, and at any time in the code I can just "submit" new As or Bs and it will trigger the conversion pipeline which in turn will trigger a job by the worker thread.

Even the resulting code looks simpler. But I still am unsure if I am abusing threads for something simple.

© Programmers or respective owner

Related posts about python

Related posts about multithreading