Even distribution through a chain of resources
Posted
by
ClosetGeek
on Programmers
See other posts from Programmers
or by ClosetGeek
Published on 2014-02-04T23:05:18Z
Indexed on
2014/06/10
3:41 UTC
Read the original article
Hit count: 179
I'm working on an algorithm which routes tasks through a chain of distributed resources based on a hash (or random number).
For example, say you have 10 gateways into a service which distribute tasks to 1000 handlers through 100 queues. 10,000 connected clients are expected to be connected to gateways at any given time (numbers are very general to keep it simple).
Thats
- 10,000 clients
- 10 gateways (producers)
- 100 queues
- 1000 workers/handlers (consumers)
The flow of each task is client->gateway->queue->worker
Each client will have it's own hash/number which is used to route each task from the client to the same worker each time, with each task going through the same gateway and queue each time. Yet the algorithm handles distribution evenly, meaning each gateway, queue, and worker will have an even workload.
My question is what exactly would this be called? Does such a thing already exist? This started off as a DHT, but I realized that DHTs can't do exactly what I need, so I started from scratch.
© Programmers or respective owner