Set LD_LIBRARY_PATH and CLASSPATH on cluster nodes before running a hadoop job

Posted by Ashish Sharma on Super User See other posts from Super User or by Ashish Sharma
Published on 2012-11-30T17:24:47Z Indexed on 2012/12/01 11:09 UTC
Read the original article Hit count: 207

Filed under:

linux

|

apache

|

hadoop

I need to set LD_LIBRARY_PATH and CLASSPATH before running a job a cluster. In LD_LIBRARY_PATH i need to add location of some jars which are required while running the job, As these jars are avaiable at my cluster, similar with CLASSPATH.

I have a 3 NODE cluster, I need to set this LD_LIBRARY_PATH and CLASSPATH for all the 3 data nodes so that the following jar are available while running the job

© Super User or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about apache

Web site not responding

as seen on Super User - Search for 'Super User'
I have website working fine before. But now its not able to connect to the server(I believe that is the problem). But its strange that the message not able to connect to the server is not coming and its keep connecting... for infinite time. Here is the screenshot. Here are some of the useful… >>> More
SVN Error 403 Forbidden

as seen on Stack Overflow - Search for 'Stack Overflow'
I can't figure this out. I try to import a new project into a svn repository from Netbeans and get 403 Forbidden. I just setup svn on my serverbox today. I can get to it through a browser just fine, though its empty as I haven't imported my project yet. Apache's path for html files is /var/www I… >>> More
How can I setup dependencies for Axis2 / Axiom on Maven2

as seen on Stack Overflow - Search for 'Stack Overflow'
I've tried the following settings on pom.xml to use Axis2 wsdl2code: <dependencies> <dependency> <groupId>org.apache.axis2</groupId> <artifactId>axis2</artifactId> <version>1.5.1</version> </dependency> </dependencies> ... <build> … >>> More
Problem compiling hive with ant

as seen on Stack Overflow - Search for 'Stack Overflow'
I compiling with Solaris 10 SPARC, jdk 1.6 from Sun, Ant 1.7.1 from OpenCSW. I have no problem running hadoop 0.17.2.1 However, I have problem compiling/integrating hive with the error 'cannot find symbol', although I followed the tutorial. I have the hive source code from SVN exactly from tutorial… >>> More
War deployment error related to classloading

as seen on Stack Overflow - Search for 'Stack Overflow'
hello when i am deploying my war file and run it it will give error like org.springframework.instrument.classloading.tomcat.TomcatInstrumentableClassLoader Jan 6, 2011 3:16:04 PM org.apache.catalina.startup.HostConfig deployDescriptor INFO: Deploying configuration descriptor servlet.xml Jan… >>> More