Error when running a basic Hadoop code

Posted by Abhishek Shivkumar on Stack Overflow See other posts from Stack Overflow or by Abhishek Shivkumar
Published on 2012-07-08T14:00:31Z Indexed on 2012/07/08 15:16 UTC
Read the original article Hit count: 193

Filed under:
|
|
|

I am running a hadoop code that has a partitioner class inside the job. But, when I run the command

hadoop jar Sort.jar SecondarySort inputdir outputdir

I am getting a runtime error that says

class KeyPartitioner not org.apache.hadoop.mapred.Partitioner.

I have ensured that the KeyPartitioner class has extended the Partitioner class, but why am I getting this error?

Here is the driver code:

JobConf conf = new JobConf(getConf(), SecondarySort.class);
    conf.setJobName(SecondarySort.class.getName());

    conf.setJarByClass(SecondarySort.class);

    conf.setInputFormat(TextInputFormat.class);
    conf.setOutputFormat(TextOutputFormat.class);

    conf.setMapOutputKeyClass(StockKey.class);
    conf.setMapOutputValueClass(Text.class);

    conf.setPartitionerClass((Class<? extends Partitioner<StockKey, DoubleWritable>>) KeyPartitioner.class);

    conf.setMapperClass((Class<? extends Mapper<LongWritable, Text, StockKey, DoubleWritable>>) StockMapper.class);
    conf.setReducerClass((Class<? extends Reducer<StockKey, DoubleWritable, Text, Text>>) StockReducer.class);

and here is the code of the partitioner class:

public class KeyPartitioner extends Partitioner<StockKey, Text> {

@Override
public int getPartition(StockKey arg0, Text arg1, int arg2) {

    int partition = arg0.name.hashCode() % arg2;

    return partition;
}
}  

© Stack Overflow or respective owner

Related posts about java

Related posts about hadoop