Global variables in hadoop.
Posted
by Deepak Konidena
on Stack Overflow
See other posts from Stack Overflow
or by Deepak Konidena
Published on 2010-05-22T16:06:40Z
Indexed on
2010/05/22
16:10 UTC
Read the original article
Hit count: 196
hadoop
Hi,
My program follows a iterative map/reduce approach. And it needs to stop if certain conditions are met. Is there anyway i can set a global variable that can be distributed across all map/reduce tasks and check if the global variable reaches the condition for completion.
Something like this.
While(Condition != true){
Configuration conf = getConf();
Job job = new Job(conf, "Dijkstra Graph Search");
job.setJarByClass(GraphSearch.class);
job.setMapperClass(DijkstraMap.class);
job.setReducerClass(DijkstraReduce.class);
job.setOutputKeyClass(IntWritable.class);
job.setOutputValueClass(Text.class);
}
Where condition is a global variable that is modified during/after each map/reduce execution.
© Stack Overflow or respective owner