Need advice on cron job'ing a very large process
Posted
by Arms
on Stack Overflow
See other posts from Stack Overflow
or by Arms
Published on 2010-03-25T00:10:16Z
Indexed on
2010/03/25
0:13 UTC
Read the original article
Hit count: 178
I have a PHP script that grabs data from an external service and saves data to my database. I need this script to run once every minute for every user in the system (of which I expect to be thousands). My question is, what's the most efficient way to run this per user, per minute? At first I thought I would have a function that grabs all the user Ids from my database, iterate over the ids and perform the task for each one, but I think that as the number of users grow, this will take longer, and no longer fall within 1 minute intervals. Perhaps I should queue the user Ids, and perform the task individually for each one? In which case, I'm actually unsure of how to proceed.
Thanks in advance for any advice.
© Stack Overflow or respective owner