一、Keepalived高可用概念
1.什么是高可用
一般是指2台机器启动着完全相同的业务系统,当有一台机器down机了,另外一台服务器就能快速的自动接管,称为高可用。对于访问的用户是无感知的。
Keepalived软件起初是专为LVS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能。因此,keepalived除了能够管理LVS软件外,还可以作为其他服务的高可用解决方案软件。
keepalived软件主要是通过VRRP协议实现高可用功能的。VRRP是Virtual Router Redundancy Protocol(虚拟路由冗余协议)的缩写,VRRP出现的目的就是为了解决静态路由的单点故障问题的,它能保证当个别节点宕机时,整个网络可以不间断地运行。所以,keepalived一方面具有配置管理LVS的功能,同时还具有对LVS下面节点进行健康检查的功能,另一方面也可以实现系统网络服务的高可用功能。
2.常用的工具
1.硬件通常使用 F5
2.软件通常使用 keepalived
3.keepalived高可用故障切换转移原理
Keepalived高可用服务对之间的故障切换转移,是通过VRRP来实现的。在keepalived服务工作时,主Master节点会不断地向备节点发送(多播的方式)心跳消息,用来告诉备Backup节点自己还活着。当主节点发生故障时,就无法发送心跳的消息了,备节点也因此无法继续检测到来自主节点的心跳了。于是就会调用自身的接管程序,接管主节点的IP资源和服务。当主节点恢复时,备节点又会释放主节点故障时自身接管的IP资源和服务,恢复到原来的备用角色
4.高可用keepalived思考问题
1.如何确定谁是主节点谁是备节点
2.如果Master故障,Backup自动接管,那么Master恢复后会夺权吗(抢占试、非抢占式)
3.如果两台服务器都认为自己是Master会出现什么问题(脑裂)
二、Nginx+Keepalived+LVS搭建
实现思路:将keepalived中的vip作为nginx负载均衡的监听地址,并且域名绑定的也是vip的地址
说明:nginx负载均衡实现高可用,需要借助keepalived地址漂移功能
1.环境准备
2.保证七层负载均衡完全一致
#下载zhihu软件,web1和web2的站点内容要一致
[root@web02 ~]# ll /code/zhihu/
total 52
drwxr-xr-x. 2 www www 72 Jun 4 2018 api
drwxr-xr-x. 37 www www 4096 Jun 4 2018 app
drwxr-xr-x. 14 www www 243 Jun 4 2018 cache
-rw-r--r--. 1 www www 17603 Jun 1 2018 changelog.txt
-rw-r--r--. 1 www www 707 Jun 12 2016 index.php
drwxr-xr-x. 3 www www 50 Jun 4 2018 install
drwxr-xr-x. 2 www www 39 Jun 4 2018 language
-rw-r--r--. 1 www www 2802 Jun 12 2016 license.txt
drwxr-xr-x. 5 www www 4096 Jun 4 2018 models
drwxr-xr-x. 4 www www 72 Jun 4 2018 plugins
-rw-r--r--. 1 www www 1885 Jun 12 2016 README.md
-rw-r--r--. 1 www www 224 Jun 12 2016 robots.txt
drwxr-xr-x. 9 www www 109 Jun 4 2018 static
drwxr-xr-x. 8 www www 4096 Jun 4 2018 system
drwxr-xr-x. 2 www www 101 Jun 1 2018 tmp
drwxr-xr-x. 2 www www 6 Jun 1 2018 uploads
-rw-r--r--. 1 www www 507 Jun 1 2018 version.php
drwxr-xr-x. 3 www www 21 Jun 1 2018 views
# web两端的zhihu配置文件要一致
[root@web01 ~]# cat /etc/nginx/conf.d/zhihu.com.conf
server {
listen 80;
server_name zhihu.com;
location / {
root /code/zhihu;
index index.php;
}
location ~* \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_param SCRIPT_FILENAME /code/zhihu/$fastcgi_script_name;
fastcgi_param HTTPS on;
include fastcgi_params;
}
}
#配置负载均衡的证书。先配置lb01,再用scp命令同步至lb02
[root@lb01 ~]# cd /etc/nginx/ssl_key/
[root@lb01 ssl_key]# ll
total 8
-rw-r--r-- 1 www www 1383 May 8 23:02 server.crt
-rw-r--r-- 1 www www 1708 May 8 23:02 server.key
#配置 负载均衡两个端的配置文件
[root@lb01 ~]# cd /etc/nginx/conf.d/
[root@lb01 conf.d]# cat zhihu.com.conf
upstream zhihu_https {
server 172.16.1.7;
server 172.16.1.8;
}
server {
listen 80;
server_name zhihu.com;
rewrite (.*) https://$server_name$1;
}
server {
listen 443 ssl;
server_name zhihu.com;
ssl_certificate /etc/nginx/ssl_key/server.crt;
ssl_certificate_key /etc/nginx/ssl_key/server.key;
location / {
proxy_pass http://zhihu_https;
include proxy_params;
}
}
#配置代理的优化文件,lb1与lb2同步
[root@lb01 nginx]# cat proxy_params
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 30;
proxy_send_timeout 60;
proxy_read_timeout 60;
proxy_buffering on;
proxy_buffer_size 32k;
proxy_buffers 4 128k;
#先访问、测试一下负载均衡必须是可用的!
3.安装keepalived
[root@lb01 ~]# yum install -y keepalived
#lb2也安装
4.配置keepalived
1)查找配置文件
[root@lb01 ~]# rpm -qc keepalived
主配置文件: /etc/keepalived/keepalived.conf
主程序文件: /usr/sbin/keepalived
file服务: keepalived.service
环境配置文件:/etc/sysconfig/keepalived
2)配置主节点的配置文件
[root@lb01 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.15.3
}
}
#解释
[root@lb01 ~]# vim /etc/keepalived/keepalived.conf
#全局配置
global_defs {
#身份识别
router_id lb01
}
#配置VRRP协议
vrrp_instance VI_1 {
#状态,MASTER和BACKUP
state MASTER
#绑定网卡
interface eth0
#虚拟路由标示,可以理解为分组
virtual_router_id 50
#优先级
priority 100
#监测心跳间隔时间
advert_int 1
#配置认证
authentication {
#认证类型
auth_type PASS
#认证的密码
auth_pass 1111
}
#设置VIP
virtual_ipaddress {
#虚拟的VIP地址
192.168.15.3
}
}
3)配置备节点
[root@lb02 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.15.3
}
}
5.启动服务
[root@lb01 ~]# systemctl start keepalived.service
[root@lb02 ~]# systemctl start keepalived.service
6.keepalived开启日志
# 查看日志存放位置
[root@lb01 ~]# tail -f /var/log/messages
#配置keepalived
[root@lb01 ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -d -S 0" # 大S
#配置rsyslog抓取日志
[root@lb01 ~]# vim /etc/rsyslog.conf
local0.* /var/log/keepalived.log
#!local0 ,0代表是配置日志里面的KEEPALIVED_OPTIONS="-D -d -S 0"末尾的数字。数字要对应
#重启服务
[root@lb01 ~]# systemctl restart keepalived.service rsyslog
三、keepalived的抢占式与 非抢占式
1.两个节点都启动的情况
#两个节点都启动时,由于节点1优先级高于节点2,所以只有节点1上有VIP
[root@lb01 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
[root@lb02 ~]# ip addr | grep 192.168.15.3
2.停止主节点
[root@lb01 ~]# systemctl stop keepalived.service
[root@lb01 ~]# ip addr | grep 192.168.15.3
#由于节点1keepalived挂掉,节点2会自动接管节点1的工作,即VIP
[root@lb02 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
3.重新启动主节点
[root@lb01 ~]# systemctl start keepalived
[root@lb01 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
#由于节点1优先级高于节点2,所以当节点1恢复时,会将VIP抢占回来
4.配置非抢占式
1)主节点配置
[root@lb01 ~]# vim /etc/keepalived/keepalived.conf
... ...
vrrp_instance VI_1 {
state BACKUP #必需把2台节点的state状态改成一样的BACKUP
nopreempt #非抢占式的命令,两个节点都必须加上配置
priority 100 #其中一个节点的优先级必须要高于另外一个节点的优先级
... ...
}
[root@lb01 ~]# systemctl restart keepalived.service
2)备节点配置
[root@lb02 ~]# vim /etc/keepalived/keepalived.conf
... ...
vrrp_instance VI_1 {
state BACKUP
nopreempt
priority 90
... ...
}
[root@lb02 ~]# systemctl restart keepalived.service
四、keepalived 脑裂
由于某些原因,导致两台keepalived高可用服务器在指定时间内,无法检测到对方的心跳,各自取得资源及服务的所有权,而此时的两台高可用服务器又都还活着。
1.脑裂的故障
1.网线松动,网络故障
2.服务器硬件故障
3.服务器之间开启了防火墙
2.脑裂模拟
1)开启防火墙
[root@lb01 ~]# systemctl start firewalld.service
[root@lb01 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
[root@lb02 ~]# systemctl start firewalld
[root@lb02 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
2)访问网站
#因为开启了firewalld防火墙,默认拒绝所有连接,要开启80端口
[root@lb01 ~]# firewall-cmd --add-service=http
success
[root@lb02 ~]# firewall-cmd --add-service=http
success
[root@lb01 ~]# firewall-cmd --add-service=https
success
[root@lb02 ~]# firewall-cmd --add-service=https
success
#访问页面没有任何问题
3)关闭防火墙
[root@lb02 ~]# systemctl stop firewalld.service
[root@lb02 ~]# ip addr | grep 192.168.15.3
[root@lb01 ~]# systemctl stop firewalld.service
[root@lb01 ~]# ip addr | grep 192.168.15.3
inet 192.168.15.3/32 scope global eth0
3.脑裂解决的办法
#如果发生闹裂,则随机kill掉一台即可
#编写检测脚本, “测试”如果能ping通主并且备节点还有VIP的话则认为产生了脑裂
[root@lb02 ~]# vim check_keepalive.sh
#!/bin/sh
vip=192.168.15.3
lb01_ip=172.16.1.5
while true;do
ssh $lb01_ip 'ip addr | grep 192.168.15.3' &>/dev/null
if [ $? -eq 0 -a `ip add|grep "$vip"|wc -l` -eq 1 ];then
echo "ha is 脑裂"
else
echo "ha is ok"
fi
sleep 3
done
五、高可用keepalived与nginx
Nginx默认监听在所有的IP地址上,VIP会飘到一台节点上,相当于那台nginx多了VIP这么一个网卡,所以可以访问到nginx所在机器
但是.....如果nginx宕机,会导致用户请求失败,但是keepalived没有挂掉不会进行切换,所以需要编写一个脚本检测Nginx的存活状态,如果不存活则kill掉keepalived
1.nginx故障切换脚本
[root@lb01 ~]# vim check_web.sh
#!/bin/sh
nginxpid=$(ps -ef | grep [n]ginx | wc -l)
if [ $nginxpid -eq 0 ];then
systemctl start nginx &>/dev/null
sleep 3
nginxpid=$(ps -ef | grep [n]ginx | wc -l)
if [ $nginxpid -eq 0 ];then
systemctl stop keepalived
fi
fi
#1.判断Nginx是否存活,如果不存活则尝试启动Nginx
#2.等待3秒后再次获取一次Nginx状态
#3.再次进行判断, 如Nginx还不存活则停止Keepalived,让地址进行漂移,并退出脚本
[root@lb01 ~]# chmod +x /root/check_web.sh #给脚本增加执行权限
2.使用keepalived配置文件调用nginx切换脚本
1)配置抢占式时
#只需要在备节点配置
[root@lb02 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
router_id lb02
}
#每5秒执行一次脚本,脚本执行内容不能超过5秒,否则会中断再次重新执行脚本
vrrp_script check_web {
script "/root/check_web.sh"
interval 5
}
vrrp_instance VI_1 {
state BACKUP
nopreempt
interface eth0
virtual_router_id 50
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.15.3
}
#调用计划的脚本
track_script {
check_web
}
}
2)配置非抢占式时
#配置非抢占式时,两边都要配置脚本
[root@lb01 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
router_id lb01
}
vrrp_script check_web {
script "/root/check_web.sh"
interval 5
}
vrrp_instance VI_1 {
state BACKUP
nopreempt
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.15.3
}
track_script {
check_web
}
}