1. 环境
一台物理服务器 9.115.251.86，上面创建两个虚机，每个虚机两个网卡：
- vm1：eth0 - 9.*.*.232 eth1：10.0.0.14
- vm2: eth0 - 9.8.*.219 eth1:10.0.0.10，上面运行DHCP Agent，管理 dnsmasq，提供 DHCP 服务
两块 eth1 连到物理机上的一个 bridge 上：
bridge name bridge id STP enabled interfaces
mgtbr0 .fa8013216b56 no tap0
vnet1
vnet12
vnet13
vnet14
vnet16
vnet18
vnet2
vnet5
2. 错误
vm1 上创建一个虚机 vmchild1，启动时发出 BOOTP 请求去向 DHCP Agent 获取 IP 地址。DHCP Agent 管理的 dnsmasq 可以正常收到 BOOTP 请求，而且发回了请求，但是 vm1 无法收到。
vm2 上 tcpdump：
::14.135874 IP (tos 0x0, ttl , id , offset , flags [none], proto UDP (), length )
10.0.0.10. > 10.0.0.14.: OTV, flags [I] (0x08), overlay , instance
IP (tos 0xc0, ttl , id , offset , flags [none], proto UDP (), length )
---.static.sonic.net.bootps > ---.static.sonic.net.bootpc: BOOTP/DHCP, Reply, length , xid 0x11f7631f, secs , Flags [none]
Your-IP ---.static.sonic.net
Server-IP ---.static.sonic.net
Client-Ethernet-Address fa::3e:ba::4c (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option , length : Offer
Server-ID Option , length : ---.static.sonic.net
Lease-Time Option , length :
RN Option , length :
RB Option , length :
Subnet-Mask Option , length : 255.255.255.0
BR Option , length : ---.static.sonic.net
Domain-Name Option , length : "openstacklocal"
Domain-Name-Server Option , length : ---.static.sonic.net
Default-Gateway Option , length : ---.static.sonic.net
Classless-Static-Route Option , length : (60.0.0.0/:0.0.0.0),(default:---.static.sonic.net)
::14.136118 IP (tos 0xc0, ttl , id , offset , flags [none], proto ICMP (), length )
10.0.0.100 > 10.0.0.10: ICMP host 10.0.0.14 unreachable - admin prohibited, length 409
IP (tos 0x0, ttl , id , offset , flags [none], proto UDP (), length )
10.0.0.10. > 10.0.0.14.: OTV, flags [I] (0x08), overlay , instance
而且发现另外的问题：
root@controller:~/s1# ping 10.0.0.13
PING 10.0.0.13 (10.0.0.13) () bytes of data.
bytes from 10.0.0.13: icmp_seq= ttl= time=0.630 ms
From 10.0.0.13: icmp_seq= Redirect Host(New nexthop: 10.0.0.13)
bytes from 10.0.0.13: icmp_seq= ttl= time=0.628 ms
From 10.0.0.13: icmp_seq= Redirect Host(New nexthop: 10.0.0.13)
3.定位
初步推断是某处防火墙阻止了 BOOTP 的包，于是将 vm1，vm2 上的防火墙全部关掉，无效。最后，查看物理机 iptables：
[root@rh65 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain
ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW tcp dpts:vnc-server:cvsup
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
发现 FORWARD 阻止了 ICMP 相关的包。
4. 解决
在物理机上运行 service iptables stop 将 iptables 关闭，问题解决。