In case to find out the cause of TCP retransmits on my Linux (RHEL, kernel 2.6.18) servers connecting to the same switch. I had a client-server pair send "Hello" to each other every 200us and captured the packets with tcpdump on the client machine. The command I used to mimic client and server are:
while [ 0 ]; do echo "Hello"; usleep 200; done | nc server 18510
while [ 0 ]; do echo "Hello"; usleep 200; done | nc -l 18510
When the server machine was busy serving some other requests, the client suffered from abrupt retransmits occasionally. But the output of tcpdump seemed irrational.
16:04:58.898970 IP server.18510 > client.34533: P 4531:4537(6) ack 3204 win 123 <nop,nop,timestamp 1923778643 3452833828>
16:04:58.901797 IP client.34533 > server.18510: P 3204:3210(6) ack 4537 win 33 <nop,nop,timestamp 3452833831 1923778643>
16:04:58.901855 IP server.18510 > client.34533: P 4537:4549(12) ack 3210 win 123 <nop,nop,timestamp 1923778646 3452833831>
16:04:58.903871 IP client.34533 > server.18510: P 3210:3216(6) ack 4549 win 33 <nop,nop,timestamp 3452833833 1923778646>
16:04:58.903950 IP server.18510 > client.34533: P 4549:4555(6) ack 3216 win 123 <nop,nop,timestamp 1923778648 3452833833>
16:04:58.905796 IP client.34533 > server.18510: P 3216:3222(6) ack 4555 win 33 <nop,nop,timestamp 3452833835 1923778648>
16:04:58.905860 IP server.18510 > client.34533: P 4555:4561(6) ack 3222 win 123 <nop,nop,timestamp 1923778650 3452833835>
16:04:58.908903 IP client.34533 > server.18510: P 3222:3228(6) ack 4561 win 33 <nop,nop,timestamp 3452833838 1923778650>
16:04:58.908966 IP server.18510 > client.34533: P 4561:4567(6) ack 3228 win 123 <nop,nop,timestamp 1923778653 3452833838>
16:04:58.911855 IP client.34533 > server.18510: P 3228:3234(6) ack 4567 win 33 <nop,nop,timestamp 3452833841 1923778653>
16:04:59.112573 IP client.34533 > server.18510: P 3228:3234(6) ack 4567 win 33 <nop,nop,timestamp 3452834042 1923778653>
16:04:59.112648 IP server.18510 > client.34533: P 4567:5161(594) ack 3234 win 123 <nop,nop,timestamp 1923778857 3452834042>
16:04:59.112659 IP client.34533 > server.18510: P 3234:3672(438) ack 5161 win 35 <nop,nop,timestamp 3452834042 1923778857>
16:04:59.114427 IP server.18510 > client.34533: P 5161:5167(6) ack 3672 win 126 <nop,nop,timestamp 1923778858 3452834042>
16:04:59.114439 IP client.34533 > server.18510: P 3672:3678(6) ack 5167 win 35 <nop,nop,timestamp 3452834044 1923778858>
16:04:59.116435 IP server.18510 > client.34533: P 5167:5173(6) ack 3678 win 126 <nop,nop,timestamp 1923778860 3452834044>
16:04:59.116444 IP client.34533 > server.18510: P 3678:3684(6) ack 5173 win 35 <nop,nop,timestamp 3452834046 1923778860>
Packet 3228:3234(6) from client was retransmitted due to ack timeout. What I could not understand was that the client machine did not send out any packets after the first 3228:3234(6) packets was sent.
The server machine had advertised a window (scaled) large enough. The data transfer up to the retransmit was fine which meant no slow start should be in action.
What can cause the client machine to stop sending until the packet timed out?
BTW, I am unable to run tcpdump on the server machine.