Out of memory while iterating through rowset

Posted by Phliplip on Stack Overflow See other posts from Stack Overflow or by Phliplip
Published on 2010-05-31T19:17:04Z Indexed on 2010/05/31 19:23 UTC
Read the original article Hit count: 255

Hi All,

I have a "small" table of 60400 rows with zipcode data. I wan't to iterate through them all, update a column value, and then save it.

The following is part of my Zipcodes model which extends My_Db_Table that a totalRows function that - you guessed it.. returns the total number of rows in the table (60400 rows)

public function normalizeTable() {
    $this->getAdapter()->setProfiler(false);

    $totalRows = $this->totalRows();        
    $rowsPerQuery = 5;

    for($i = 0; $i < $totalRows; $i = $i + $rowsPerQuery) {
        $select = $this->select()->limit($i, $rowsPerQuery);

        $rowset = $this->fetchAll($select);
        foreach ($rowset as $row) {
            $row->{self::$normalCityColumn} = $row->normalize($row->{self::$cityColumn});
            $row->save();
        }
        unset($rowset);
    }
}

My rowClass contains a normalize function (basicly a metaphone wrapper doing some extra magic).

At first i tried a plain old $this->fetchAll(), but got a out of memory (128MB) right away. Then i tried splitting the rowset into chunks, only difference is that some rows actually gets updated.

Any ideas on how i can acomplish this, or should i fallback to ye'olde mysql_query()

© Stack Overflow or respective owner

Related posts about zend-framework

Related posts about memory-allocation