php parsing csv with ftell

Posted by Robert82 on Stack Overflow See other posts from Stack Overflow or by Robert82
Published on 2013-11-02T03:34:53Z Indexed on 2013/11/02 3:53 UTC
Read the original article Hit count: 130

Filed under:
|

I have a 500mb csv file with over 500,000 lines, each with 80 fields. I am using fget to process the file line by line.

$col1 = array();
while (($row = fgetcsv($handle, 1000, ",")) !== FALSE) {
  $col1[] = $row[0];
}

Because of an execution time limit on the PHP file by my hosting provider (120 seconds), I can't process the whole file in one run.

I tried using ftell() and fseek() to remember the last position for restart. The trouble is, sometimes the ftell() position is in the middle of a row, and resuming means missing the first half of the row.

Is there an elegant way to know the last line successfully processed, and resume from the one after it? I realize I can do a simple counter, and then loop through to that point again, but that would produce diminishing returns on the rows I can process towards the end of the file.

Is there something like ftell() and fseek() that would work in my case? Or a way to limit ftell() to return the pointer for the end of the previous line?

© Stack Overflow or respective owner

Related posts about php

Related posts about csv