I've been bashing my head against a wall with this one all day and I would greatly appreciate a few more eyes on the problem at hand.

We have an in-house SVN Server that contains all live and development code for our website. Our live server can connect to this and get updates from the repository.

This was all working fine until we migrated the SVN Server from a physical machine to a vSphere VM. Now, for some reason that continues to fathom me, we can no longer connect to the SVN Server.

The SVN Server runs CentOS 6.2, Apache and SVN 1.7.2. SELinux is well and trully disabled and the problem remains when iptables is stopped.

Our production server does run an older version of CentOS and SVN but the same system worked previously so I don't think that this is the issue.

Of note, if I have iptables enabled, using service iptables status, I can see a single packet coming in and being accepted but the production server simply hangs on any svn command. If I give up waiting and do a CTRL-C to break the process I get a "could not connect to server".

To me it appears to be something to do with the SVN Server rejecting external connections but I have no idea how this would happen.

Any thoughts on what I can try from here?

Thanks, Rob

Edit: Network topology Production server sits externally to our in-house SVN server. Our IPCop (?) firewall allows connections from it (and it alone) on port 80 and passes the connection to the SVN Server. The hardware is all pretty decent and I don't doubt that its doing its job correctly, especially as iptables is seeing the new connections.

subversion.conf (in /etc/httpd/conf.d)

LoadModule dav_svn_module     modules/mod_dav_svn.so
<Location /repos>
   DAV svn
   SVNPath /var/svn/repos

   <LimitExcept PROPFIND OPTIONS REPORT>
      AuthType Basic
      AuthName "SVN Server"
      AuthUserFile /var/svn/svn-auth
      Require valid-user
   </LimitExcept>
</Location>

ifconfig

eth0      Link encap:Ethernet  HWaddr 00:0C:29:5F:C8:3A
          inet addr:172.16.0.14  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe5f:c83a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:32317 errors:0 dropped:0 overruns:0 frame:0
          TX packets:632 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2544036 (2.4 MiB)  TX bytes:143207 (139.8 KiB)

netstat -lntp

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      1484/mysqld
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      1135/rpcbind
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1351/sshd
tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      1230/cupsd
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1575/master
tcp        0      0 0.0.0.0:58401               0.0.0.0:*                   LISTEN      1153/rpc.statd
tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      1626/qpidd
tcp        0      0 :::139                      :::*                        LISTEN      1678/smbd
tcp        0      0 :::111                      :::*                        LISTEN      1135/rpcbind
tcp        0      0 :::80                       :::*                        LISTEN      1615/httpd
tcp        0      0 :::22                       :::*                        LISTEN      1351/sshd
tcp        0      0 ::1:631                     :::*                        LISTEN      1230/cupsd
tcp        0      0 ::1:25                      :::*                        LISTEN      1575/master
tcp        0      0 :::445                      :::*                        LISTEN      1678/smbd
tcp        0      0 :::56799                    :::*                        LISTEN      1153/rpc.statd

iptables --list -v -n (when iptables is stopped)

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

iptables --list -v -n (when iptables is running, after one attempted svn connection)

Chain INPUT (policy ACCEPT 68 packets, 6561 bytes)
 pkts bytes target     prot opt in     out     source               destination
   19  1304 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22
    1    60 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:80
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:80
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:80

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 17 packets, 1612 bytes)
 pkts bytes target     prot opt in     out     source               destination

tcpdump

17:08:18.455114 IP 'production server'.43255 > 'svn server'.local.http: Flags [S], seq 3200354543, win 5840, options [mss 1380,sackOK,TS val 2011458346 ecr 0,nop,wscale 7], length 0
17:08:18.455169 IP 'svn server'.local.http > 'production server'.43255: Flags [S.], seq 629885453, ack 3200354544, win 14480, options [mss 1460,sackOK,TS val 816478 ecr 2011449346,nop,wscale 7], length 0
17:08:19.655317 IP 'svn server'.local.http > 'production server'k.43255: Flags [S.], seq 629885453, ack 3200354544, win 14480, options [mss 1460,sackOK,TS val 817679 ecr 2011449346,nop,wscale 7], length 0