Network traffic is not being forwarded from a VM to the network using a bridged interface with Xen + libvirt
- by foob
I'm having trouble getting network access from a VM that I'm running using Xen and libvirt. I've been trying different things and reading similar posts online for a couple of days but I'm really stuck at this point. If anybody could offer some insight it would be much appreciated.
I have a VM that I'm running on a host with a bridge set up as br0 and an interface eth0 on a 192.168.60.0/24 subnet. The networking portion of the libvirt configuration xml is:
<interface type='bridge'>
<mac address='ff:a0:d1:e5:07:de'/>
<source bridge='br0'/>
<script path='/etc/xen/scripts/vif-bridge'/>
<model type='virtio' />
</interface>
When I start the VM a vif6.0 interface is created on the host and the ifconfig output is:
br0 Link encap:Ethernet HWaddr 00:A0:D1:C3:07:DE
inet addr:192.168.60.33 Bcast:192.168.60.255 Mask:255.255.255.0
inet6 addr: fe80::2a0:d1ff:fee5:7de/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3570 (3.4 KiB) TX bytes:3508 (3.4 KiB)
eth0 Link encap:Ethernet HWaddr 00:A0:D1:C3:07:DE
inet6 addr: fe80::2a0:d1ff:fee5:7de/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:492 (492.0 b)
Interrupt:19 Memory:fe8f0000-fe900000
vif6.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:80 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:6660 (6.5 KiB) TX bytes:468 (468.0 b)
virbr0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
The 'brctl show' output seems to show the bridge being configured correctly:
br0 8000.00a0d1e507de no eth0
vif6.0
The ifcfg-eth0 contents in the VM are:
DEVICE=eth0
BOOTPROTO=static
HWADDR=FF:A0:D1:E5:07:DE
IPADDR=192.168.60.133
NETMASK=255.255.255.0
ONBOOT=yes
and the output of ifconfig in the VM look like what I would expect:
eth0 Link encap:Ethernet HWaddr FF:A0:D1:E5:07:DE
inet addr:192.168.60.133 Bcast:192.168.60.255 Mask:255.255.255.0
inet6 addr: fe80::fda0:d1ff:fee5:7de/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:80 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:468 (468.0 b) TX bytes:7780 (7.5 KiB)
but when I try to ssh or ping another computer I get 'no route to host.'
Using tcpdump on the host system I tried to see if I could narrow down where the problem is:
# tcpdump -vv -i vif6.0
tcpdump: WARNING: vif6.0: no IPv4 address assigned
tcpdump: listening on vif6.0, link-type EN10MB (Ethernet), capture size 96 bytes
14:49:40.833997 arp who-has 192.168.60.35 tell 192.168.60.133
14:49:41.833314 arp who-has 192.168.60.35 tell 192.168.60.133
14:49:42.833309 arp who-has 192.168.60.35 tell 192.168.60.133
So the VM is sending out out an arp who-has packet when I try to ssh to 192.168.60.35. I think that this means the setup within the VM is ok and that this is an issue on the host system. If I run tcpdump with the interface of br0 then I don't see these arp packets.
My thought here is that the packets are being blocked before going on to the bridge somehow. I tried adding an iptables rule to resolve this:
-A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
but it didn't work. I also tried the following:
/sbin/sysctl -w net.bridge.bridge-nf-call-ip6tables=0
/sbin/sysctl -w net.bridge.bridge-nf-call-iptables=0
/sbin/sysctl -w net.bridge.bridge-nf-call-arptables=0
/sbin/sysctl -w net.ipv4.ip_forward=1
which had no impact.
Is it obvious to somebody who has more experience than me what I'm missing here? Should vif6.0 have the same MAC address is eth0 in the vm? Do I need more rules in my iptables? Thanks for any help!
