Faster forking of large processes on Linux ?

Posted by timday on Stack Overflow See other posts from Stack Overflow or by timday
Published on 2010-04-28T16:56:05Z Indexed on 2010/04/28 18:07 UTC
Read the original article Hit count: 279

Filed under:

fork

|

exec

|

processes

|

linux

|

debian

What's the fastest, best way on modern Linux of achieving the same effect as a fork-execve combo from a large process ?

My problem is that the process forking is ~500MByte big, and a simple benchmarking test achieves only about 50 forks/s from the process (c.f ~1600 forks/s from a minimally sized process) which is too slow for the intended application.

Some googling turns up vfork as having being invented as the solution to this problem... but also warnings about not to use it. Modern Linux seems to have acquired related clone and posix_spawn calls; are these likely to help ? What's the modern replacement for vfork ?

I'm using 64bit Debian Lenny on an i7 (the project could move to Squeeze if posix_spawn would help).

© Stack Overflow or respective owner

Related posts about fork

fork() within a fork()

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Is there any way to differentiate the child processes created by different fork() functions within a program. global variable i; SIGCHLD handler function() { i--; } handle() { fork() --> FORK2 } main() { while(1) { if(i<5) { i++; if( (fpid=fork())==0)… >>> More
Fork bomb protection not working : Amount of processes not limited

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I just came to realize that my system is not limiting the amount of processes per user properly thus not preventing a user from dring a fork-bomb and crashing the entire system: user@thebe:~$ cat /etc/security/limits.conf | grep user user hard nproc 512 user@thebe:~$ ulimit -u 1024 user@thebe:~$… >>> More
fork() in perl on windows

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm using fork() on PERL in windows (activeperl) for a basic socket server, but apparently there are problems (it won't accept connections after a few times), is there any workaround? while($client = $bind->accept()) { $client->autoflush(); if(fork()){ $client->close(); } else… >>> More
Behavior of a pipe after a fork()

as seen on Stack Overflow - Search for 'Stack Overflow'
When reading about pipes in Advanced Programming in the UNIX Environment, I noticed that after a fork that the parent can close() the read end of a pipe and it doesn't close the read end for the child. When a process forks, does its file descriptors get retained? What I mean by this is that before… >>> More
How does fork() return for child process

as seen on Stack Overflow - Search for 'Stack Overflow'
I know that fork() returns differently for the son and father processes, but I'm unable to find information on how this happens. How does the son process receive the return value 0 from fork? And what is the difference in regards to the call stack? As I understand it, for the father it goes something… >>> More

Related posts about exec

Error during maven/ant build: "[java] Timestamp response not valid"

as seen on Stack Overflow - Search for 'Stack Overflow'
My maven build started failing randomly, and it got the following error which I cannot make sense of, and googling it doesn't give me anything useful: [echo] Creating a full package... [java] Timestamp response not valid [INFO] ------------------------------------------------------------------------ [ERROR]… >>> More
CCNet and .Net 4.0

as seen on Stack Overflow - Search for 'Stack Overflow'
Could not load file or assembly 'System.Data' or one of its dependencies. An attempt was made to load a program with an incorrect format We have just upgraded one of our projects to .Net 4.0 and have re-configured our build server (Cruise Control .Net). Anyone help with the above error?… >>> More
in x64 Windows is there a way to run a Runtime.exec() process avoiding 'Registry redirection'

as seen on Stack Overflow - Search for 'Stack Overflow'
Our app runs in jvm 32 bit, even when in windows x64. Now, at some point, I need to access some registry values, for example HKEY_LOCAL_MACHINE/SOFTWARE/mycomp. I do this by executing cmd /C reg query HKEY_LOCAL_MACHINE\SOFTWARE\mycop from Runtime.exec() and parsing the output. This works fine… >>> More
Uploading a file through exec, hanging script

as seen on Stack Overflow - Search for 'Stack Overflow'
Basically in a nutshell is I am using exec() to run WinSCP, which is causing the script to hold until the file is uploaded. Is there anyway to make it so that the script continues, and the upload runs in the background? Running PHP 5.3.1 on Win7. >>> More
PHP exec() on windows not working

as seen on Stack Overflow - Search for 'Stack Overflow'
Hey, I'm trying to execute a program on windows through PHP, the command is posted below. This doesn't seem to be running through the script at all, even though it works when the command is manually entered into the command prompt. exec('C:\\ffmpeg -i ' . $movedfile . ' -acodec aac -ab 128k -vcodec… >>> More