Reading a simple Avro file from HDFS
Posted
by
John Galt... who
on Stack Overflow
See other posts from Stack Overflow
or by John Galt... who
Published on 2012-07-24T13:36:54Z
Indexed on
2013/11/05
3:55 UTC
Read the original article
Hit count: 174
I am trying to do a simple read of an Avro file stored in HDFS. I found out how to read it when it is on the local file system....
FileReader reader = DataFileReader.openReader(new File(filename), new GenericDatumReader());
for (GenericRecord datum : fileReader) {
String value = datum.get(1).toString();
System.out.println("value = " value);
}
reader.close();
My file is in HDFS, however. I cannot give the openReader a Path or an FSDataInputStream. How can I simply read an Avro file in HDFS?
EDIT: I got this to work by creating a custom class (SeekableHadoopInput) that implements SeekableInput. I "stole" this from "Ganglion" on github. Still, seems like there would be a Hadoop/Avro integration path for this.
Thanks
© Stack Overflow or respective owner