Streaming large result sets with MySQL
Posted
by configurator
on Stack Overflow
See other posts from Stack Overflow
or by configurator
Published on 2010-03-15T13:16:19Z
Indexed on
2010/03/15
13:19 UTC
Read the original article
Hit count: 188
I'm developing a spring application that uses large MySQL tables. When loading large tables, I get an OutOfMemoryException
, since the driver tries to load the entire table into application memory.
I tried using
statement.setFetchSize(Integer.MIN_VALUE);
but then every ResultSet I open hangs on close()
; looking online I found that that happens because it tries loading any unread rows before closing the ResultSet, but that is not the case since I do this:
ResultSet existingRecords = getTableData(tablename);
try {
while (existingRecords.next()) {
// ...
}
} finally {
existingRecords.close(); // this line is hanging, and there was no exception in the try clause
}
The hangs happen for small tables (3 rows) as well, and if I don't close the RecordSet (which happened in one method) then connection.close()
hangs.
© Stack Overflow or respective owner