Network traffic is not being forwarded from a VM to the network using a bridged interface with Xen + libvirt

Posted by foob on Server Fault See other posts from Server Fault or by foob
Published on 2013-04-24T19:21:08Z Indexed on 2013/10/26 3:57 UTC
Read the original article Hit count: 534

Filed under:
|
|
|
|

I'm having trouble getting network access from a VM that I'm running using Xen and libvirt. I've been trying different things and reading similar posts online for a couple of days but I'm really stuck at this point. If anybody could offer some insight it would be much appreciated.

I have a VM that I'm running on a host with a bridge set up as br0 and an interface eth0 on a 192.168.60.0/24 subnet. The networking portion of the libvirt configuration xml is:

<interface type='bridge'>
  <mac address='ff:a0:d1:e5:07:de'/>
  <source bridge='br0'/>
  <script path='/etc/xen/scripts/vif-bridge'/>
  <model type='virtio' />
</interface>

When I start the VM a vif6.0 interface is created on the host and the ifconfig output is:

br0       Link encap:Ethernet  HWaddr 00:A0:D1:C3:07:DE
          inet addr:192.168.60.33  Bcast:192.168.60.255  Mask:255.255.255.0
          inet6 addr: fe80::2a0:d1ff:fee5:7de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3570 (3.4 KiB)  TX bytes:3508 (3.4 KiB)

eth0      Link encap:Ethernet  HWaddr 00:A0:D1:C3:07:DE
          inet6 addr: fe80::2a0:d1ff:fee5:7de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:492 (492.0 b)
          Interrupt:19 Memory:fe8f0000-fe900000

vif6.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:80 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:6660 (6.5 KiB)  TX bytes:468 (468.0 b)

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

The 'brctl show' output seems to show the bridge being configured correctly:

br0             8000.00a0d1e507de       no              eth0
                                                        vif6.0

The ifcfg-eth0 contents in the VM are:

DEVICE=eth0 
BOOTPROTO=static 
HWADDR=FF:A0:D1:E5:07:DE
IPADDR=192.168.60.133 
NETMASK=255.255.255.0 
ONBOOT=yes

and the output of ifconfig in the VM look like what I would expect:

eth0      Link encap:Ethernet  HWaddr FF:A0:D1:E5:07:DE  
          inet addr:192.168.60.133  Bcast:192.168.60.255  Mask:255.255.255.0
          inet6 addr: fe80::fda0:d1ff:fee5:7de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:80 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:468 (468.0 b)  TX bytes:7780 (7.5 KiB)

but when I try to ssh or ping another computer I get 'no route to host.'

Using tcpdump on the host system I tried to see if I could narrow down where the problem is:

# tcpdump -vv -i vif6.0
tcpdump: WARNING: vif6.0: no IPv4 address assigned
tcpdump: listening on vif6.0, link-type EN10MB (Ethernet), capture size 96 bytes
14:49:40.833997 arp who-has 192.168.60.35 tell 192.168.60.133
14:49:41.833314 arp who-has 192.168.60.35 tell 192.168.60.133
14:49:42.833309 arp who-has 192.168.60.35 tell 192.168.60.133

So the VM is sending out out an arp who-has packet when I try to ssh to 192.168.60.35. I think that this means the setup within the VM is ok and that this is an issue on the host system. If I run tcpdump with the interface of br0 then I don't see these arp packets.

My thought here is that the packets are being blocked before going on to the bridge somehow. I tried adding an iptables rule to resolve this: -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT but it didn't work. I also tried the following:

/sbin/sysctl -w net.bridge.bridge-nf-call-ip6tables=0
/sbin/sysctl -w net.bridge.bridge-nf-call-iptables=0
/sbin/sysctl -w net.bridge.bridge-nf-call-arptables=0
/sbin/sysctl -w net.ipv4.ip_forward=1

which had no impact.

Is it obvious to somebody who has more experience than me what I'm missing here? Should vif6.0 have the same MAC address is eth0 in the vm? Do I need more rules in my iptables? Thanks for any help!

© Server Fault or respective owner

Related posts about networking

Related posts about iptables