What is the best way to iterate over a large result set in JDBC/iBatis 3?

Posted by paul_sns on Stack Overflow See other posts from Stack Overflow or by paul_sns
Published on 2010-04-27T10:28:22Z Indexed on 2010/04/28 12:53 UTC
Read the original article Hit count: 352

Filed under:
|
|

We're trying to iterate over a large number of rows from the database and convert those into objects. Behavior will be as follows:

  • Result will be sorted by sequence id, a new object will be created when sequence id changes. The object created will be sent to an external service and will sometimes have to wait before sending another one (which means the next set of data will not be immediately used)
  • We already have invested code in iBatis 3 so an iBatis solution will be the best approach for us (we've tried using RowBounds but haven't seen how it does the iteration under the hood).
  • We'd like to balance minimizing memory usage and reducing number of DB trips.
  • We're also open to pure JDBC approach but we'd like the solution to work on different databases.

UPDATE:

  • We need to make as few calls to DB as possible (1 call would be the ideal scenario) while also preventing the application to use too much memory. Are there any other solutions out there for this type of problem may it be pure JDBC or any other technology?

Thanks and hope to hear your insights on this.

© Stack Overflow or respective owner

Related posts about java

Related posts about jdbc