Multiple servers acting like a single one with all the hardware?

Posted by marc.riera on Server Fault See other posts from Server Fault or by marc.riera
Published on 2010-01-26T22:34:53Z Indexed on 2010/03/11 18:19 UTC
Read the original article Hit count: 315

Filed under:

gridengine

|

linux

|

clustering

Hello,

by now I have 10 servers for hpc, power computing oriented. My users need to launch several processes using qmake. The users are used to work with ubuntu 9.10, and the software from the repositories is switable for them.

I've deployed ubuntu 9.10 to all 10 servers (pxe rocks).

By now we work with parallel-ssh and cluster-ssh, which allows as to launch the same process to all servers. With this tools this tools the servers remain as independent but with the same software and the same launched command.

Now we would like to go to next step and see all the servers as a single one with all the resources from the other 9 as if was its resources.

The difference would be substantial in time to process and also time to design the command to launch.

Any advice on wich software to use will be very useful?

Thanks

© Server Fault or respective owner

Related posts about gridengine

Garbled UI with vnc, chicken-of-the-vnc, gridengine, qmon

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Ubuntu 11.04 gridengine version 6.2u5-1ubuntu1 I start a desktop with the vncserver command. My ~/.vnc/xstartup ends with 'gnome-session'. I connect to it using chicken-of-the-VNC on my MacOSX Lion system. I run 'qmon'. Much of qmon works, but several critical tasks show hopelessly garbled grid… >>> More
Is there a way to tell SGE to run specific jobs as root on the execution node?

as seen on Server Fault - Search for 'Server Fault'
The title kinda says it all... We're using SGE/OGE to submit jobs to a set of worker nodes that then do things with specific pieces of equipment. The programs and scripts that have been created that manipulate this equipment rely on running as root. I'd like SGE to handle allocation of resources… >>> More
SGE: invoking qmake raises "critical error: can't resolve group"

as seen on Server Fault - Search for 'Server Fault'
I'm new to SGE an I'm trying to run qmake with the simple following Makefile with our very new cluster: merge.txt: job1.txt job2.txt job3.txt ... cat $^ > $@ job1.txt: sleep 1 echo "Hello From " $@ > $@ sleep 1 job2.txt: sleep 2 echo "Hello From " $@ > $@ sleep… >>> More
SGE - limit a user to a certain host, using resource quota configuration

as seen on Server Fault - Search for 'Server Fault'
Is it possible to limit a user to a particular host, using the Resource Quota Configuration option in qmon for Sun Grid Engine? I'm thinking of a line to the effect of: { ... limit users {john} to hostname=compute-1-1.local } The documentation mentions built in resource types: slots, arch, mem_total… >>> More
Parallel prologue and epilogue in Grid Engine

as seen on Server Fault - Search for 'Server Fault'
We have a cluster being used to run MPI jobs for a customer. Previously this cluster used Torque as the scheduler, but we are transitioning to Grid Engine 6.2u5 (for some other features). Unfortunately, we are having trouble duplicating some of our maintenance scripts in the Grid Engine environment… >>> More

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More