Perl Parallel::ForkManager wait_all_children() takes excessively long time

Posted by zhang18 on Stack Overflow See other posts from Stack Overflow or by zhang18
Published on 2010-03-11T21:08:44Z Indexed on 2010/03/11 21:09 UTC
Read the original article Hit count: 341

I have a script that uses Parallel::ForkManager. However, the wait_all_children() process takes incredibly long time even after all child-processes are completed. The way I know is by printing out some timestamps (see below). Does anyone have any idea what might be causing this (I have 16 CPU cores on my machine)?

my $pm = Parallel::ForkManager->new(16)
for my $i (1..16) {
    $pm->start($i) and next;

    ... do something within the child-process ...

    print (scalar localtime), " Process $i completed.\n";
    $pm->finish();
}
print (scalar localtime), " Waiting for some child process to finish.\n"; 
$pm->wait_all_children();
print (scalar localtime), " All processes finished.\n"; 

Clearly, I'll get the Waiting for some child process to finish message first, with a timestamp of, say, 7:08:35. Then I'll get a list of Process i completed messages, with the last one at 7:10:30. However, I do not receive the message All Processes finished until 7:16:33(!). Why is that 6-minute delay between 7:10:30 and 7:16:33? Thx!

© Stack Overflow or respective owner

Related posts about perl

Related posts about parallel-processing