Any frameworks or library allow me to run large amount of concurrent jobs schedully?
- by Yoga
Are there any high level programming frameworks that allow me to run large amount of concurrent jobs schedully?
e.g. I have 100K of urls need to check their uptime every 5 minutes
Definitely I can write a program to handle this, but then I need to handle concurrency, queuing, error handling, system throttling, job distribution etc.
Will there be a framework that I only focus on a particular job (i.e. the ping task) and the system will take care of the scaling and error handling for me?
I am open to any language.