Problem with a large CSV file

Posted by moustafa on Stack Overflow See other posts from Stack Overflow or by moustafa
Published on 2010-05-26T15:42:53Z Indexed on 2010/05/26 16:01 UTC
Read the original article Hit count: 109

Filed under:

I have a very large CSV file. 51427 lines to be exact.

I need to import the entire file into a MySQL database, however, the script times out due to server settings and slow connection (and maybe other reasons that I am not aware of).

So - I am now passing parameters START and LIMIT via address bar to import, like this:

http://my.server.address/import.php?...000&limit=1000

This reads the entire CSV file into an array, and starts at line 10000 of the array and inserts into the database until it reaches line 11000, and then terminates the script.

This works very nicely, however, I am not happy having to reach the entire 51427 lines of the CSV file into an array before processing.

Is there not a way where I can only read the required lines into an array? That would speed things up significantly.

© Stack Overflow or respective owner

Related posts about php