Low coupling processing big quantities of data
Posted
by
vitalik
on Programmers
See other posts from Programmers
or by vitalik
Published on 2012-04-09T09:05:19Z
Indexed on
2012/04/09
11:47 UTC
Read the original article
Hit count: 453
Usually I achieve low coupling by creating classes that exchange lists, sets, and maps between them. Now I am developing a batch application and I can't put all the data inside a data structure because there isn't enough memory. I have to read and process one chunk of data and then going to the next one. So having low coupling is much more difficult because I have to check somewhere if there is still data to read, etc.
What I am using now is:
Source -> Process -> Persist
The classes that process have to ask to the Source classes if there are more rows to read.
What are the best practices and or useful patterns in such situations?
I hope I am explaining myself, if not tell me.
© Programmers or respective owner