Kubernetes网络通信：
(1) 容器间通信：同一个Pod内的多个容器间的通信, lo
(2) Pod通信：Pod IP <--> Pod IP
(3) Pod与Service通信：PodIP <--> ClusterIP
(4) Service与集群外部客户端的通信；
CNI:
flannel
calico
canel
kube-router
... 解决方案：
虚拟网桥
多路复用：MacVLAN
硬件交换：SR-IOV
[root@master ~]# cat /etc/cni/net.d/-flannel.conflist
{
"name": "cbr0",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
flannel：
支持多种后端：
VxLAN
() vxlan
() Directrouting
host-gw: Host Gateway #不推荐，只能在二层网络中，不支持跨网络，如果有成千上万的Pod，容易产生广播风暴
UDP: 性能差 flannel的配置参数：
Network：flannel使用的CIDR格式的网络地址，用于为Pod配置网络功能；
10.244.0.0/ ->
master: 10.244.0.0/
node01: 10.244.1.0/
...
node255: 10.244.255.0./ 10.0.0.0/
10.0.0.0/
...
10.255.255.0/ SubnetLen：把Network切分子网供各节点使用时，使用多长的掩码进行切分，默认为24位； SubnetMin：10.244.10.0/ SubnetMax: 10.244.100.0/ Backend：vxlan, host-gw, udp
vxlan:
flannel Pod间跨主机通信原理抓包[root@master ~]# ip rdefault via 172.20.0.1 dev ens3 proto dhcp metric
10.244.0.0/ dev cni0 proto kernel scope link src 10.244.0.1
10.244.1.0/ via 10.244.1.0 dev flannel. onlink
10.244.2.0/ via 10.244.2.0 dev flannel. onlink
172.17.0.0/ dev docker0 proto kernel scope link src 172.17.0.1
172.20.0.0/ dev ens3 proto kernel scope link src 172.20.0.91 metric
从路由可以看出 Pod cni0 -----> flannel.1 -----> 物理网卡 -----> 对端
[root@master ~]# ip a
: lo: <LOOPBACK,UP,LOWER_UP> mtu qdisc noqueue state UNKNOWN group default qlen
link/loopback ::::: brd :::::
inet 127.0.0.1/ scope host lo
valid_lft forever preferred_lft forever
: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc pfifo_fast state UP group default qlen
link/ether ::::b2:ca brd ff:ff:ff:ff:ff:ff
inet 172.20.0.91/ brd 172.20.255.255 scope global noprefixroute dynamic ens3
valid_lft 2412sec preferred_lft 2412sec
: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue state UP group default
link/ether :::::b4 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/ brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
: flannel.: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue state UNKNOWN group default
link/ether :e1::::c1 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.0/ scope global flannel.
valid_lft forever preferred_lft forever
: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue state UP group default qlen
link/ether 0a::0a:f4:: brd ff:ff:ff:ff:ff:ff
inet 10.244.0.1/ scope global cni0
valid_lft forever preferred_lft forever
: veth0f580b07@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master cni0 state UP group default
link/ether a2:::9b:b5:dc brd ff:ff:ff:ff:ff:ff link-netnsid
: vethb8510761@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master cni0 state UP group default
link/ether 1a::6b:::fc brd ff:ff:ff:ff:ff:ff link-netnsid
: vethfc114a8b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master cni0 state UP group default
link/ether fa:ec:f9:ee:: brd ff:ff:ff:ff:ff:ff link-netnsid
: veth023640f@if30028: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master docker0 state UP group default
link/ether d2:8a::e6:8f: brd ff:ff:ff:ff:ff:ff link-netnsid
: br-973161700d44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue state UP group default
link/ether :::2e:ec: brd ff:ff:ff:ff:ff:ff
inet 172.18.0.1/ brd 172.18.255.255 scope global br-973161700d44
valid_lft forever preferred_lft forever
: vethfdee34f@if13226: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master docker0 state UP group default
link/ether 7e:3e:6a:e2::f0 brd ff:ff:ff:ff:ff:ff link-netnsid
: veth49aa01b@if13228: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu qdisc noqueue master br-973161700d44 state UP group default
link/ether b2:::6e:4d:da brd ff:ff:ff:ff:ff:ff link-netnsid
cni0 提供Pod网络共享的地址来源
flannel.1 是对数据包进行 vxlan封装
[root@master ~]# brctl show cni0
bridge name bridge id STP enabled interfaces
cni0 .0a580af40001 no veth0f580b07
vethb8510761
vethfc114a8b
[root@master ~]# brctl show flannel.
bridge name bridge id STP enabled interfaces
flannel. can't get info Operation not supported
抓包
15:13:26.796845 IP 172.20.0.93.52954 > 172.20.0.76.otv: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.1.175 > 10.244.2.223: ICMP echo request, id 3072, seq 221, length 64