Any good method for mounting Hadoop HDFS from another system?

Posted by Beel on Server Fault See other posts from Server Fault or by Beel
Published on 2009-08-15T04:56:35Z Indexed on 2010/03/12 2:07 UTC
Read the original article Hit count: 299

Filed under:

mount

|

hadoop

I want to mount the Cloudera Hadoop as a Linux file system over the LAN.

As a setup, I already have the hadoop cluster running on a set of Ubuntu machines. But now I need to be able to use it as a normal file system from a Fedora system over the LAN.
I tried FUSe but two things:
1. Cloudera says FUSE loses data (click here for that comment by a Cloudera employee on the official Cloudera support site)
2. I've had no success making it work the way we want

As a point of clarification, I am using Hadoop ONLY for the file system, not for its other capabilities.

© Server Fault or respective owner

Related posts about mount

12.10 update breaks NFS mount

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've just upgraded to the latest 12.10 beta. Rebooted twice. The problem is with the NFS folders not mounting, here's a verbose log. # mount -v myserver:/nfs_shared/tools /tools/ mount: no type was given - I'll assume nfs because of the colon mount.nfs: timeout set for Mon Oct 1 11:42:28 2012 mount… >>> More
Mount SMB / AFP 13.10

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I cannot seem to get Ubuntu to mount a mac share via SMB or AFP. I've tried the following... AFP: apt-get install afpfs-ng-utils mount_afp afp://user:password@localip/share /mnt/share Error given: "Could not connect, never got a reponse to getstatus, Connection timed out". Which is odd as I can… >>> More
Mount Return Code for CIFS mount

as seen on Server Fault - Search for 'Server Fault'
When I run the following command (as root or via sudo) from a bash script I get an exit status (or return code in mount man page parlance) of 1: mount -v -t cifs //nasbox/volume /tmpdir/ --verbose -o credentials=/root/cifsid & /tmp/mylog It outputs the following into the myflog file: parsing… >>> More
Disable raid member check upon mount to mount damaged nvidia raid1 member

as seen on Server Fault - Search for 'Server Fault'
Hi, A friend of mine destroyed his Nvidia RAID1 array somehow and in trying to fix it, he ended up with a non-working array. Because of the RAID metadata, the actual disk data was stored at an offset from the beginning. I was able to identify this offset with dd and a hexeditor and then I used losetup… >>> More
Network shares do not mount.

as seen on Super User - Search for 'Super User'
My network shares were mounting fine yesterday.. suddenly they are not. They were mounting fine for the last two weeks or however long since I added them. When I run sudo mount -a I get the following error: topsy@monolyth:~$ sudo mount -a mount error(12): Cannot allocate memory Refer to the mount… >>> More

Related posts about hadoop

prerequisites of learnig hadoop, can php developer learn hadoop without java experience [closed]

as seen on Programmers - Search for 'Programmers'
i am willing to learn hadoop as a Developer , but i am confused over the prerequisite of learning it.? is having a good experience in java programming very essential to learn hadoop? I have 4 years of experience in application development in LAMP. But i am not in touch with java programming as a part… >>> More
Hadoop hdfs namenode is throwing an error

as seen on Server Fault - Search for 'Server Fault'
Full list of error: hb@localhost:/etc/hadoop/conf$ sudo service hadoop-hdfs-namenode start * Starting Hadoop namenode: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-localhost.out 12/09/10 14:41:09 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG:… >>> More
Combining HBase and HDFS results in Exception in makeDirOnFileSystem

as seen on Server Fault - Search for 'Server Fault'
Introduction An attempt to combine HBase and HDFS results in the following: 2014-06-09 00:15:14,777 WARN org.apache.hadoop.hbase.HBaseFileSystem: Create Dir ectory, retries exhausted 2014-06-09 00:15:14,780 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. java… >>> More
Problem compiling hive with ant

as seen on Stack Overflow - Search for 'Stack Overflow'
I compiling with Solaris 10 SPARC, jdk 1.6 from Sun, Ant 1.7.1 from OpenCSW. I have no problem running hadoop 0.17.2.1 However, I have problem compiling/integrating hive with the error 'cannot find symbol', although I followed the tutorial. I have the hive source code from SVN exactly from tutorial… >>> More
no namenode error in pseudo-mode

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm new to hadoop and is in learning phase. As per Hadoop Definitve guide, i have set up my hadoop in pseudo distributed mode and everything was working fine. I was even able to execute all the examples from chapter 3 yesterday. Today, when i rebooted my unix and tried to run start-dfs.sh and then… >>> More