Keepalived+Nginx实现高可用负载均衡集群

2021-07-03 22:39:48

一环境介绍

1.操作系统
CentOS Linux release 7.2.1511 (Core)

2.服务
keepalived+nginx双主高可用负载均衡集群及LAMP应用
keepalived-1.2.13-7.el7.x86_64
nginx-1.10.2-1.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64

二原理及拓扑图

1.vrrp协议
在现实的网络环境中，两台需要通信的主机大多数情况下并没有直接的物理连接。对于这样的情况，它们之间路由怎样选择？主机如何选定到达目的主机的下一跳路由，这个问题通常的解决方法有二种：
在主机上使用动态路由协议(RIP、OSPF等)
在主机上配置静态路由
很明显，在主机上配置动态路由是非常不切实际的，因为管理、维护成本以及是否支持等诸多问题。配置静态路由就变得十分流行，但路由器(或者说默认网关default gateway)却经常成为单点故障。VRRP的目的就是为了解决静态路由单点故障问题，VRRP通过一竞选(election)协议来动态的将路由任务交给LAN中虚拟路由器中的某台VRRP路由器。

2.nginx反代
nginx是以反向代理的方式进行负载均衡的。反向代理（Reverse Proxy）方式是指以代理服务器来接受Internet上的连接请求，然后将请求转发给内部网络上的服务器，并将从服务器上得到的结果返回给Internet上请求连接的客户端，此时代理服务器对外就表现为一个服务器。(为了理解反向代理，这里插播一条什么是正向代理：正向代理指的是，一个位于客户端和原始服务器之间的服务器，为了从原始服务器取得内容，客户端向代理发送一个请求并指定目标(原始服务器)，然后代理向原始服务器转交请求并将获得的内容返回给客户端。)
3.拓扑图

三配置

1.后端RS配置

1 2	`[root@inode4 ~]# yum install httpd -y` `[root@inode5 ~]# yum install httpd -y`

2.Nginx反代配置
MASTER:

upstream websrvs {

server 172.18.67.11:80;

server 172.18.67.12:80;

server 127.0.0.1:80 backup;

}

server {

listen 80 ;

location / {

proxy_pass http://websrvs;

}

BACKUP:

upstream websrvs {

server 172.18.67.11:80;

server 172.18.67.12:80;

server 127.0.0.1:80 backup;

}

server {

listen 80 ;

location / {

proxy_pass http://websrvs;

}

3.keepalived高可用配置
MASTER:

! Configuration File for keepalived

global_defs {

notification_email {

root@localhost

}

notification_email_from keepalived@localhost

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id node1

vrrp_mcast_group4 224.0.67.67

}

vrrp_script chk_down {

script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"

interval 1

weight -5

}

vrrp_script chk_nginx {

script "killall -0 nginx && exit 0 || exit 1"

interval 1

weight -5

fall 2

rise 1

}

vrrp_instance myr {

state MASTER

interface eno16777736

virtual_router_id 167

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 571f97b2

}

virtual_ipaddress {

172.18.67.33/16 dev eno16777736

}

track_script {

chk_down

chk_nginx

}

notify_master "/etc/keepalived/notify.sh master"

notify_backup "/etc/keepalived/notify.sh backup"

notify_fault "/etc/keepalived/notify.sh fault"

}

BACKUP:

! Configuration File for keepalived

global_defs {

notification_email {

root@localhost

}

notification_email_from keepalived@localhost

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id node1

vrrp_mcast_group4 224.0.67.67

}

vrrp_script chk_down {

script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"

interval 1

weight -5

}

vrrp_script chk_nginx {

script "killall -0 nginx && exit 0 || exit 1"

interval 1

weight -5

fall 2

rise 1

}

vrrp_instance myr {

state BACKUP

interface eno16777736

virtual_router_id 167

priority 95

advert_int 1

authentication {

auth_type PASS

auth_pass 571f97b2

}

virtual_ipaddress {

172.18.67.33/16 dev eno16777736

}

track_script {

chk_down

chk_nginx

}

notify_master "/etc/keepalived/notify.sh master"

notify_backup "/etc/keepalived/notify.sh backup"

notify_fault "/etc/keepalived/notify.sh fault"

}

4.通知脚本示例

[root@inode2 nginx]# vim notify.sh

#!/bin/bash

#

contact='root@localhost'

notify() {

mailsubject="$(hostname) to be $1, vip floating"

mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"

echo "$mailbody" | mail -s "$mailsubject" $contact

}

case $1 in

master)

notify master

;;

backup)

notify backup

;;

fault)

notify fault

;;

*)

echo "Usage: $(basename $0) {master|backup|fault}"

exit 1

;;

esac

节点二同样配置

四启动服务并测试

1.启动后端web服务器

1 2	`[root@inode4 ~]# systemctl start httpd` `[root@inode5 ~]# systemctl start httpd`

为了测试显示效果明显一点，自定义一个访问页面

1 2	`[root@inode4 ~]# echo "RS1:172.18.67.11" > /var/www/html/index.html` `[root@inode5 ~]# echo "RS2:172.18.67.12" > /var/www/html/index.html`

2.测试
MASTER:

[root@inode2 ~]# systemctl start keepalived

[root@inode2 ~]# systemctl status -l keepalived

● keepalived.service - LVS and VRRP High Availability Monitor

Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)

Active: active (running) since Mon 2017-05-15 15:45:20 CST; 3s ago

Process: 20971 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)

Main PID: 20972 (keepalived)

CGroup: /system.slice/keepalived.service

├─20972 /usr/sbin/keepalived -D

├─20973 /usr/sbin/keepalived -D

└─20974 /usr/sbin/keepalived -D

May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Opening file '/etc/keepalived/keepalived.conf'.

May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Configuration is using : 7521 Bytes

May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Using LinkWatch kernel netlink reflector...

May 15 15:45:20 inode2 Keepalived_vrrp[20974]: VRRP_Script(chk_nginx) succeeded

May 15 15:45:21 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Transition to MASTER STATE

May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Entering MASTER STATE

May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) setting protocol VIPs.

May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33

May 15 15:45:22 inode2 Keepalived_vrrp[20974]: Opening script file /etc/keepalived/notify.sh

May 15 15:45:22 inode2 Keepalived_healthcheckers[20973]: Netlink reflector reports IP 172.18.67.33 added

[root@inode2 ~]# ip a l

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:0c:29:8b:08:6f brd ff:ff:ff:ff:ff:ff

inet 172.18.67.13/16 brd 172.18.255.255 scope global eno16777736

valid_lft forever preferred_lft forever

inet 172.18.67.33/16 scope global secondary eno16777736

valid_lft forever preferred_lft forever

inet6 fe80::20c:29ff:fe8b:86f/64 scope link tentative dadfailed

valid_lft forever preferred_lft forever

主节点启动，Entering MASTER STATE，此时我们在客户端进行测试访问

[root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;done

RS1:172.18.67.11

RS2:172.18.67.12

RS1:172.18.67.11

RS2:172.18.67.12

访问正常，接下来我们启动备用节点的服务器

BACKUP:

[root@inode3 keepalived]# systemctl start keepalived

[root@inode3 keepalived]# systemctl status -l keepalived

● keepalived.service - LVS and VRRP High Availability Monitor

Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)

Active: active (running) since Mon 2017-05-15 15:46:51 CST; 3s ago

Process: 24329 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)

Main PID: 24330 (keepalived)

CGroup: /system.slice/keepalived.service

├─24330 /usr/sbin/keepalived -D

├─24331 /usr/sbin/keepalived -D

└─24332 /usr/sbin/keepalived -D

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering Kernel netlink command channel

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering gratuitous ARP shared channel

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening file '/etc/keepalived/keepalived.conf'.

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Configuration is using : 66427 Bytes

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Using LinkWatch kernel netlink reflector...

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering BACKUP STATE

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening script file /etc/keepalived/notify.sh

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeeded

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeeded

[root@inode3 keepalived]# ip a l

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff

inet 172.18.67.14/16 brd 172.18.255.255 scope global eno16777736

valid_lft forever preferred_lft forever

inet6 fe80::20c:29ff:fe78:24c3/64 scope link tentative dadfailed

valid_lft forever preferred_lft forever

此时，我们可以看到备用节点服务器启动后进入了BACKUP状态，Entering BACKUP STATE。接下来我们测试主节点宕机的情形下，我们的服务是否还可用

1	`[root@inode2 ~]# systemctl stop keepalived`

主节点宕机后我们查看备用节点的状态

[root@inode3 keepalived]# systemctl status -l keepalived

● keepalived.service - LVS and VRRP High Availability Monitor

Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)

Active: active (running) since Mon 2017-05-15 15:46:51 CST; 2min 19s ago

Process: 24329 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)

Main PID: 24330 (keepalived)

CGroup: /system.slice/keepalived.service

├─24330 /usr/sbin/keepalived -D

├─24331 /usr/sbin/keepalived -D

└─24332 /usr/sbin/keepalived -D

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeeded

May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeeded

May 15 15:48:35 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Transition to MASTER STATE

May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering MASTER STATE

May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) setting protocol VIPs.

May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33

May 15 15:48:36 inode3 Keepalived_vrrp[24332]: Opening script file /etc/keepalived/notify.sh

May 15 15:48:36 inode3 Keepalived_healthcheckers[24331]: Netlink reflector reports IP 172.18.67.33 added

May 15 15:48:41 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736 for 172.18.67.33

[root@inode3 keepalived]# ip a l

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff

inet 172.18.67.14/16 brd 172.18.255.255 scope global eno16777736

valid_lft forever preferred_lft forever

inet 172.18.67.33/16 scope global secondary eno16777736

valid_lft forever preferred_lft forever

inet6 fe80::20c:29ff:fe78:24c3/64 scope link tentative dadfailed

valid_lft forever preferred_lft forever

我们发现备用节点由备用状态进入了主状态，并且IP地址也成功绑定至备用节点下。再次进行测试访问

[root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;done

RS1:172.18.67.11

RS2:172.18.67.12

RS1:172.18.67.11

RS2:172.18.67.12

测试一台web服务器宕机

[root@inode4 ~]# systemctl stop httpd

[root@inode1 ~]# for i in {1..4};do curl http://172.18.67.33;done

RS2:172.18.67.12

在实际生产环境中后端两台web服务器的内容应该一样的，在这里我们可认为客户端已成功访问到服务器，因此我们可认为这样的架构体现了高可用负载均衡。

码农公寓

一 环境介绍

二 原理及拓扑图

三 配置

四 启动服务并测试

相关文章

一环境介绍

二原理及拓扑图

三配置

四启动服务并测试