Low coupling processing big quantities of data

Posted by vitalik on Programmers See other posts from Programmers or by vitalik
Published on 2012-04-09T09:05:19Z Indexed on 2012/04/09 11:47 UTC
Read the original article Hit count: 453

Usually I achieve low coupling by creating classes that exchange lists, sets, and maps between them. Now I am developing a batch application and I can't put all the data inside a data structure because there isn't enough memory. I have to read and process one chunk of data and then going to the next one. So having low coupling is much more difficult because I have to check somewhere if there is still data to read, etc.

What I am using now is:

Source -> Process -> Persist

The classes that process have to ask to the Source classes if there are more rows to read.

What are the best practices and or useful patterns in such situations?

I hope I am explaining myself, if not tell me.

© Programmers or respective owner

Related posts about design-patterns

Related posts about programming-practices