lvs nat模式+iptables实现fullnat

2023-12-09 23:20:51

一、服务器信息

lvs服务器：10.30.20.90

nginx服务器：10.30.8.64,10.30.12.71

vip：10.30.20.253

二、部署

1、nginx服务器部署

# systemctl start docker
# docker pull nginx
# mkdir /data/html
# IP=$(ifconfig eth0|awk -F'[ ]+' '/inet/ {print $3}')
# echo "${IP}" >/data/html/index.html
# docker run -d --network=host -v /data/html:/usr/share/nginx/html nginx
# curl "http://127.0.0.1"
10.30.8.64

2、部署lvs nat模式

# yum install -y ipvsadm ipset conntrack
# ip address add 10.30.20.253/32 dev eth0
# ipvsadm -A -t 10.30.20.253:8181 -s rr
# ipvsadm -a -t 10.30.20.253:8181 -r 10.30.8.64:80 -m
# ipvsadm -a -t 10.30.20.253:8181 -r 10.30.12.71:80 -m
# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.30.20.253:8181 rr
  -> 10.30.8.64:80                Masq    1      0          0         
  -> 10.30.12.71:80               Masq    1      0          0         
# sysctl net.ipv4.ip_forward=1

3、iptables配置

# iptables -t nat -N KUBE-SERVICES
# iptables -t nat -N KUBE-MARK-MASQ
# iptables -t nat -N KUBE-POSTROUTING
# iptables -t nat -N KUBE-NODE-PORT
# ipset create KUBE-NODE-PORT-TCP bitmap:port range 0-65535 
# ipset add KUBE-NODE-PORT-TCP 8181 
# iptables -t nat -A PREROUTING -j KUBE-SERVICES
# iptables -t nat -A KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT
# iptables -t nat -A KUBE-NODE-PORT -p tcp -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-MARK-MASQ
# iptables -t nat -A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
# iptables -t nat -A POSTROUTING -j KUBE-POSTROUTING
# iptables -t nat -A KUBE-POSTROUTING -m mark --mark 0x4000/0x4000 -j MASQUERADE
# sysctl net.ipv4.vs.conntrack=1

4、测试

# curl "http://10.30.20.253:8181"
10.30.12.71
# curl "http://10.30.20.253:8181"
10.30.8.64

三、抓包

1、在nginx服务器

15:07:10.282569 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [S], seq 3207943442, win 65535, options [mss 1388,nop,wscale 5,nop,nop,TS val 226579042 ecr 0,sackOK,eol], length 0
15:07:10.282650 IP 10.30.8.64.80 > 10.30.20.90.55888: Flags [S.], seq 1357773976, ack 3207943443, win 28960, options [mss 1460,sackOK,TS val 2630379 ecr 226579042,nop,wscale 7], length 0
15:07:10.289414 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [.], ack 1, win 4128, options [nop,nop,TS val 226579051 ecr 2630379], length 0
15:07:10.290442 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [P.], seq 1:82, ack 1, win 4128, options [nop,nop,TS val 226579051 ecr 2630379], length 81: HTTP: GET / HTTP/1.1
15:07:10.290470 IP 10.30.8.64.80 > 10.30.20.90.55888: Flags [.], ack 82, win 227, options [nop,nop,TS val 2630386 ecr 226579051], length 0
15:07:10.290849 IP 10.30.8.64.80 > 10.30.20.90.55888: Flags [P.], seq 1:236, ack 82, win 227, options [nop,nop,TS val 2630387 ecr 226579051], length 235: HTTP: HTTP/1.1 200 OK
15:07:10.290974 IP 10.30.8.64.80 > 10.30.20.90.55888: Flags [P.], seq 236:247, ack 82, win 227, options [nop,nop,TS val 2630387 ecr 226579051], length 11: HTTP
15:07:10.298179 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [.], ack 236, win 4120, options [nop,nop,TS val 226579058 ecr 2630387], length 0
15:07:10.298297 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [.], ack 247, win 4120, options [nop,nop,TS val 226579058 ecr 2630387], length 0
15:07:10.298306 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [F.], seq 82, ack 247, win 4120, options [nop,nop,TS val 226579059 ecr 2630387], length 0
15:07:10.298443 IP 10.30.8.64.80 > 10.30.20.90.55888: Flags [F.], seq 247, ack 83, win 227, options [nop,nop,TS val 2630394 ecr 226579059], length 0
15:07:10.305622 IP 10.30.20.90.55888 > 10.30.8.64.80: Flags [.], ack 248, win 4120, options [nop,nop,TS val 226579065 ecr 2630394], length 0

2、在lvs服务器

# conntrack -L|grep 55888
conntrack v1.4.4 (conntrack-tools): 12 flow entries have been shown.
tcp      6 111 TIME_WAIT src=10.127.12.138 dst=10.30.20.253 sport=55888 dport=8181 src=10.30.8.64 dst=10.30.20.90 sport=80 dport=55888 [ASSURED] mark=0 use=1

SNAT：10.127.12.138:55888 --> 10.30.20.90.55888 DNAT：10.30.20.253:8181 --> 10.30.8.64.80

参考文档：

https://www.jianshu.com/p/89f126b241db https://blog.dianduidian.com/post/lvs-snat%E5%8E%9F%E7%90%86%E5%88%86%E6%9E%90/

码农公寓

相关文章