Reverse massive text file in Java
Posted
by DanJanson
on Stack Overflow
See other posts from Stack Overflow
or by DanJanson
Published on 2010-04-27T23:35:04Z
Indexed on
2010/04/28
0:03 UTC
Read the original article
Hit count: 264
What would be the best approach to reverse a large text file that is uploaded asynchronously to a servlet that reverses this file in a scalable and efficient way?
- text file can be massive (gigabytes long)
- can assume mulitple server/clustered environment to do this in a distributed manner.
- open source libraries are encouraged to consider
I was thinking of using Java NIO to treat file as an array on disk (so that I don't have to treat the file as a string buffer in memory). Also, I am thinking of using MapReduce to break up the file and process it in separate machines.
Any input is appreciated. Thanks.
- Daniel
© Stack Overflow or respective owner