Ok, this is a really weird one.
I have an MPI program, where each process has to generate random numbers in a fixed range (the range is read from file). What happens is that even though I seed each process with a different value, and the numbers generated by rand() are different in each process, the expression to generate the random numbers still yields the same sequence between them.
Here's all relevant code:
// 'rank' will be unique for each process
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
// seed the RNG with a different value for each process
srand(time(NULL) + rank);
// print some random numbers to see if we get a unique sequence in each process
// 'log' is a uniquely named file, each process has its own
log << rand() << " " << rand() << " " << rand() << std::endl;
// do boring deterministic stuff
while (true)
{
// waitTimeMin and waitTimeMax are integers, Max is always greater than Min
waitSecs = waitTimeMin + rand() % (waitTimeMax - waitTimeMin);
log << "waiting " << waitSecs << " seconds" << std::endl;
sleep(waitSecs);
// do more boring deterministic stuff
}
Here's the output of each process, with 3 processes generating numbers in the range [1,9].
process 1:
15190 28284 3149
waiting 6 seconds
waiting 8 seconds
waiting 9 seconds
waiting 4 seconds
process 2:
286 6264 3153
waiting 6 seconds
waiting 8 seconds
waiting 9 seconds
waiting 4 seconds
process 3:
18151 17013 3156
waiting 6 seconds
waiting 8 seconds
waiting 9 seconds
waiting 4 seconds
So while rand() clearly generates different numbers, the expression to calculate waitSecs still evaluates to the same sequence on all processes. What's even weirder: if I run the program with the same parameteres again, only the first 3 random numbers will change, the rest of the "random" sequence will be exactly the same in each run! Changing the range of numbers will obviously produce a different result from this one, but the same parameters always yield the same sequence, between processes and between executions: except for the first 3 numbers.
Just what the hell is going on here?