Even distribution through a chain of resources
- by ClosetGeek
I'm working on an algorithm which routes tasks through a chain of distributed resources based on a hash (or random number).
For example, say you have 10 gateways into a service which distribute tasks to 1000 handlers through 100 queues. 10,000 connected clients are expected to be connected to gateways at any given time (numbers are very general to keep it simple).
Thats
10,000 clients
10 gateways (producers)
100 queues
1000 workers/handlers (consumers)
The flow of each task is client-gateway-queue-worker
Each client will have it's own hash/number which is used to route each task from the client to the same worker each time, with each task going through the same gateway and queue each time. Yet the algorithm handles distribution evenly, meaning each gateway, queue, and worker will have an even workload.
My question is what exactly would this be called? Does such a thing already exist? This started off as a DHT, but I realized that DHTs can't do exactly what I need, so I started from scratch.
