Nginx 搭建LVS-DR模式

1 环境准备

1.1 服务器与IP规划

LVS服务器
VIP(虚拟IP):192.168.51.103
DIP(转发者IP/内网IP):192.168.51.4
Nginx服务器两台(RealServer)
RIP(真实IP/内网IP):192.168.51.5
RIP(真实IP/内网IP):192.168.51.6

1.2 停止NetworkManager服务

三台服务器分别执行以下两行命令,因为服务器使用的是虚拟机,需要停止 NetworkManager,不然会出现问题

[root@localhost network-scripts]# systemctl stop NetworkManager
[root@localhost network-scripts]# systemctl disable NetworkManager

2 配置LVS节点与ipvsadm

2.1 创建子接口

LVS服务器

[root@localhost network-scripts]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# ls
ifcfg-ens33      ifcfg-lo     ifdown-eth   ifdown-isdn  ifdown-routes  ifdown-TeamPort  ifup-aliases  ifup-ippp  ifup-plip   ifup-ppp     ifup-Team      ifup-wireless      network-functions-ipv6
ifdown       ifdown-ippp  ifdown-post  ifdown-sit     ifdown-tunnel    ifup-bnep     ifup-ipv6  ifup-plusb  ifup-routes  ifup-TeamPort  init.ipv6-global
ifcfg-ens33.bak  ifdown-bnep  ifdown-ipv6  ifdown-ppp   ifdown-Team    ifup             ifup-eth      ifup-isdn  ifup-post   ifup-sit     ifup-tunnel    network-functions
[root@localhost network-scripts]# cp ifcfg-ens33 ifcfg-ens33:1

2.2 修改配置文件

[root@localhost network-scripts]# vi ifcfg-ens33:1
BOOTPROTO=static
DEVICE=ens33:1
ONBOOT=yes
IPADDR=192.168.51.103
NETMASK=255.255.255.0

2.3 刷新配置

[root@localhost network-scripts]# service network restart

2.4 安装ipvsadm

  • 好像阿里云不支持虚拟IP,购买官网的负载均衡
  • 好像腾讯云支持最大虚拟IP数量为10
ipvsadm -Ln

[root@localhost network-scripts]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

3 为两台RS配置虚拟IP

两台RS机器的配置相同,下面的操作需要在两台机器上都进行操作

3.1 复制配置

[root@localhost network-scripts]# cd /etc/sysconfig/network-scripts
[root@localhost network-scripts]# ls
ifcfg-ens33       ifdown-eth   ifdown-isdn  ifdown-routes  ifdown-TeamPort  ifup-aliases  ifup-ippp  ifup-plip   ifup-ppp     ifup-Team      ifup-wireless      network-functions-ipv6
ifcfg-ens33.bak  ifdown       ifdown-ippp  ifdown-post  ifdown-sit     ifdown-tunnel    ifup-bnep     ifup-ipv6  ifup-plusb  ifup-routes  ifup-TeamPort  init.ipv6-global
ifcfg-lo         ifdown-bnep  ifdown-ipv6  ifdown-ppp   ifdown-Team    ifup             ifup-eth      ifup-isdn  ifup-post   ifup-sit     ifup-tunnel    network-functions
[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:1

3.2 修改配置

[root@localhost network-scripts]# vi ifcfg-lo:1
DEVICE=lo:1
IPADDR=192.168.51.103
NETMASK=255.255.255.255
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback

3.3 刷新配置

方式一

[root@localhost network-scripts]# ifup lo

方式二

[root@localhost network-scripts]# service network restart

4 为两台RS配置arp

4.1 arp介绍

ARP响应级别与通告行为 。

1.arp-ignore:ARP响应级别(处理请求)

  • 0:只要本机配置了ip,就能响应请求
  • 1:请求的目标地址到达对应的网络接口,才会响应请求

2.arp-announce:ARP通告行为(返回响应)

  • 0:本机上任何网络接口都向外通告,所有的网卡都能接受到通告
  • 1:尽可能避免本网卡与不匹配的目标进行通告
  • 2:只在本网卡通告

4.2 修改配置文件

[root@localhost network-scripts]# vi /etc/sysctl.conf
# configration for lvs
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2

4.3 刷新配置

[root@localhost network-scripts]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2

4.4 增加网关

[root@localhost network-scripts]# route add -host 192.168.51.103 dev lo:1

设置开机添加

[root@localhost network-scripts]# echo "route add -host 192.168.51.103 dev lo:1" >> /etc/rc.local

5 使用ipvsadm配置集群规则

5.1 创建LVS节点

用户访问数据的集群调度者

[root@localhost network-scripts]# ipvsadm -A -t 192.168.51.103:80 -s rr -p 5
  • -A:添加集群
  • -t:tcp协议
  • ip地址:设定集群的访问ip,也就是LVS的虚拟ip
  • -s:设置负载均衡的算法,rr表示轮询
  • -p:设置连接持久化的时间

5.2 创建2台RS真实服务器

[root@localhost network-scripts]# ipvsadm -a -t 192.168.51.103:80 -r 192.168.51.5:80 -g
[root@localhost network-scripts]# ipvsadm -a -t 192.168.51.103:80 -r 192.168.51.6:80 -g

5.3 保存到规则库

不保存,重启会失效

[root@localhost network-scripts]# ipvsadm -S

5.4 检查集群

查看集群列表

[root@localhost network-scripts]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.51.103:80 rr persistent 5
  -> 192.168.51.5:80              Route   1      0          0         
  -> 192.168.51.6:80              Route   1      0          0    

查看集群状态

[root@localhost network-scripts]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  192.168.51.103:80                   0        0        0        0        0
  -> 192.168.51.5:80                     0        0        0        0        0
  -> 192.168.51.6:80                     0        0        0        0        0

5.5 其他命令

# 重启ipvsadm,重启后需要重新配置
service ipvsadm restart
# 查看持久化连接
ipvsadm -Ln --persistent-conn
# 查看连接请求过期时间以及请求源ip和目标ip
ipvsadm -Lnc
# 设置tcp tcpfin udp 的过期时间(一般保持默认)
ipvsadm --set 1 1 1
# 查看过期时间
ipvsadm -Ln --timeout

详细的帮助文档

ipvsadm -h
man ipvsadm

6 相关信息

  • 博文不易,辛苦各位猿友点个关注和赞,感谢
上一篇:Windows下 git bash无法使用conda解决办法


下一篇:关于Windows部署scrapyd项目报错的两个问题