Map Reduce Frameworks/Infrastructure
Posted
by Johannes Rudolph
on Stack Overflow
See other posts from Stack Overflow
or by Johannes Rudolph
Published on 2010-03-15T20:35:29Z
Indexed on
2010/03/15
20:39 UTC
Read the original article
Hit count: 346
Map Reduce is a pattern that seems to get a lot of traction lately and I start to see it manifest in one of my projects that is focused on an event processing pipeline (iPhone Accelerometer and GPS data). I needed to built a lot of infrastructure for this project, in fact it overweighs the logic code interacting with it by 2x. Some of the components I built where EventProcessors (with in- and output plus buffering, timing etc.), multiplexers and aggregators.
This leads me to my question what the "common" required infrastrucutre for map reduce is. Since I am working with .Net a lot I can see map reduce infrastructure built into the Framework and language constructs. Functional languages support this paradigm per se. It seems every language can be used with map reduce, some have better support than others, others again are built around that concept (e.g. Go). And there are Frameworks like Apache Hadoop to support map reduce.
© Stack Overflow or respective owner