Get info from multiple files, match it and then display to end user, what is fastest?
- by Patrick
Hi,
I need to build a website where we display data that is refreshed every 5minutes
in a text file with a | separator.
I currently use Java to do this.
What I do now:
I grab the textfile for every request through the website and process it and then display
the data to the end user, this works fine, since Java can go through like 5000 lines of data fast, and when I filter it it is still extremely fast.
However now the management wants the following:
They added 3 textfiles with the | separator to it, and now want me to also read
those files and match the information on certain fields, and if there is a match
also display that information to the end user.
I think soon enough, although Java is fast, I will run into trouble when 10 people
want that information and I have to run through 4 total files matching the information.
What can I do to make this process super fast?
My Creative solutions so far:
-Leave it this way, since Java is fast and end users can wait (probably less then 1second)
-Have a background process that dumps new data into a MySQL database every 5minutes, since databases are extremely good at getting the same data from multiple tables.
Thank you!