keepalived实现高可用

集群Cluster

集群类型:

LB    lvs/nginx(http/upstream, stream/upstream)
HA    高可用性
SPoF: Single Point of Failure
HPC

系统可用性的公式:A=MTBF/(MTBF+MTTR)

(0,1), 95%
几个9(指标): 99%, ..., 99.999%,99.9999%

系统故障:

硬件故障:设计缺陷、wear out(损耗)、自然灾害……
软件故障:设计缺陷

   提升系统高用性的解决方案之降低MTTR

手段:冗余redundant
active/passive 主备
active/active 双主
active --> HEARTBEAT --> passive
active <--> HEARTBEAT <--> active

高可用的是“服务”

HA nginx service:
       vip/nginx process[/shared storage]
资源:组成一个高可用服务的“组件”
(1) passive node的数量
(2) 资源切换

    shared storage:

NAS:文件共享服务器;
SAN:存储区域网络,块级别的共享

Network partition:网络分区

quorum:法定人数
    with quorum: > total/2
    without quorum: <= total/2
隔离设备: fence
    node:STONITH = Shooting The Other Node In The Head
        断电重启
    资源:断开存储的连接

   TWO nodes Cluster

    辅助设备:ping node, quorum disk

Failover:故障切换,即某资源的主节点故障时,将资源转移至其它节点的操作
Failback:故障移回,即某资源的主节点故障后重新修改上线后,将之前已转移至其它节点的资源重新切回的过程

HA Cluster实现方案:

AIS:应用接口规范 完备复杂的HA集群
  RHCS:Red Hat Cluster Suite红帽集群套件
  heartbeat
  corosync
vrrp协议实现:虚拟路由冗余协议,解决静态网关单点风险
  软件层—keepalived
  物理层—路由器、三层交换机

KeepAlived

keepalived:

   vrrp协议:Virtual Router Redundancy Protocol

术语:

虚拟路由器:Virtual Router
虚拟路由器标识:VRID(0-255),唯一标识虚拟路由器
物理路由器:
  master:主设备
  backup:备用设备
  priority:优先级
VIP:Virtual IP
VMAC:Virutal MAC (00-00-5e-00-01-VRID)

通告:心跳,优先级等;周期性

工作方式:抢占式,非抢占式

安全工作:

认证:
无认证
简单字符认证:预共享密钥
MD5

工作模式:

主/备:单虚拟路由器
主/主:主/备(虚拟路由器1),备/主(虚拟路由器2)

keepalived:

    vrrp协议的软件实现,原生设计目的为了高可用ipvs服务

功能:

vrrp协议完成地址流动
为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
为ipvs集群的各RS做健康状态检测
基于脚本调用接口通过执行脚本完成脚本中定义的功能,进而影响集群事务,以此支持nginx、haproxy等服务

  组件:

核心组件:

vrrp stack
ipvs wrapper
checkers

控制组件:配置文件分析器

IO复用器
内存管理组件

 KeepAlived组成 

keepalived实现高可用

KeepAlived实现

HA Cluster 配置准备:

(1) 各节点时间必须同步

ntp, chrony

(2) 确保iptables及selinux不会成为阻碍

(3) 各节点之间可通过主机名互相通信(对KA并非必须)

建议使用/etc/hosts文件实现

(4) 各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信(对KA并非必须)

   Keepalived安装:

   keepalived包,CentOS 6.4+ Base源(直接可以在本地yum源仓库直接安装)

程序环境:

主配置文件:/etc/keepalived/keepalived.conf
主程序文件:/usr/sbin/keepalived
Unit File:/usr/lib/systemd/system/keepalived.service
Unit File的环境配置文件:/etc/sysconfig/keepalived

实验一:实现keepalived主从方式高可用实战:

架构图:

keepalived实现高可用

搭建环境:

类型

机器名称

IP配置

服务角色

A

client

192.168.43.11

客户端访问(桥接模式)

B

router

左:192.168.43.13

右:192.168.37.106

左路由器(桥接模式)

右路由器(NAT模式)

C

lvs-server1

VIP:192.168.37.100

DIP:192.168.37.27

负载均衡调度器(NAT模式)

D

lvs-server2

VIP:192.168.37.100

DIP:192.168.37.37

负载均衡调度器(NAT模式)

E

rs1

RIP:192.168.37.19

VIP:192.168.37.100

GW:192.168.37.106

后端服务器(NAT模式)

F

rs2

RIP:192.168.37.20

VIP:192.168.37.100

GW:192.168.37.106

后端服务器(NAT模式)


原理:主从:一主一从,主服务器的在工作,从服务器的在待命状态;主的宕机了,VIP漂移到从服务器上,由从提供服务

  1、在LVS1和LVS2服务器上,直接用本地yum源安装keepalived包

[root@ka1~]#yum install keepalived -y
[root@ka2~]#yum install keepalived -y

 2、在LVS1修改keepalived配置文件之前先备份一份,以防万一配置问题,无法还原默认值。

[root@ka1~]#cd /etc/keepalived
[root@ka1keepalived]#cp keepalived.conf{,.bak}

 3、在配置之前先实现基于key验证,并将双方的hosts文件写入对方配置文件中,方便LVS1和LVS2主机解析。

[root@ka1~]#ssh-keygen
[root@ka1~]#ssh-copy-id 192.168.37.37
[root@ka2~]#ssh-keygen
[root@ka2~]#ssh-copy-id 192.168.37.27

 在ka1主机修改hosts配置文件,并将此配置文件传到ka2.

[root@ka1~]#cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.37.27  ka1
192.168.37.37  ka2
[root@ka1~]#scp /etc/hosts 192.168.37.37:/etc/

  4、在客户端上配置网关和本机的IP地址

[root@centos6 network-scripts]# cat ifcfg-eth0 
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.1.11
GATEWAY=192.168.1.13
PREFIX=24
DNS1=114.114.114.114

 5、在路由器上开启路由转发

[root@router~]#vim /etc/sysctl.conf 开启路由转发功能
net.ipv4.ip_forward=1  
[root@router~]#sysctl -p  生效路由配置文件

    在路由器左接口外网(桥接模式)添加一个IP地址

[root@routernetwork-scripts]#cat ifcfg-ens37
DEVICE=ens37
BOOTPROTO=none
IPADDR=192.168.1.13
PREFIX=24
ONBOOT=yes
DNS1=114.114.114.114

   在路由器右接口内网(NAT模式)添加一个IP地址

[root@routernetwork-scripts]#cat ifcfg-ens33
DEVICE=ens33
BOOTPROTO=none
IPADDR=192.168.37.106
PREFIX=24
ONBOOT=yes
DNS1=114.114.114.114

  6、配置RS1服务器的IP地址,网关指向路由器右边的IP地址

[root@rs1network-scripts]#cat ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.19
PREFIX=24
GATEWAY=192.168.37.106
DNS1=114.114.114.114
ONBOOT=yes
BOOTPROTO=static

 7、配置RS2服务器的IP地址,网关指向路由器有边的IP地址

[root@rs2network-scripts]#cat ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.20
BOOTPROTO=static
GATEWAY=192.168.37.106
PREFIX=24
DNS1=114.114.114.114
ONBOOT=yes

   8、在LVS1服务器上添加一个VIP地址和DIP地址,并将网关指向路由器右端的IP地址

[root@lvsnetwork-scripts]#cat  ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.27    DIP地址
PREFIX=24
BOOTPROTO=static
GATEWAY=192.168.37.106   指向路由器右端IP地址(192.168.37.106)的网关
ONBOOT=yes
DNS1=114.114.114.114

  9、在LVS2服务器上添加一个VIP地址和DIP地址,并将网关指向路由器右端的IP地址

[root@lvsnetwork-scripts]#cat  ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.37    DIP地址
PREFIX=24
BOOTPROTO=static
GATEWAY=192.168.37.106   指向路由器右端IP地址(192.168.37.106)的网关
ONBOOT=yes
DNS1=114.114.114.114

配置ka主节点的keeplived文件 

3、配置ka1机器的keepalived文件。

[root@ka1]cd /etc/keepalived
[root@ka1keepalived]#vim keepalived.conf 
global_defs {   修改全局的配置
   notification_email {
       root@localhost   故障通知邮件信息
   }
   notification_email_from  keepalived@localhost
   smtp_server 172.0.0.1   添加本机的smpt地址,发邮件
   smtp_connect_timeout 30
   router_id ka1  本机主机名
   #vrrp_mcast_group4 244.100.100.100  添加多播地址(D网段),多播地址有冲突
}

vrrp_instance VI_1 {  自定义示例名称
    state MASTER      当前虚拟化路由器的初始化状态 MASTER|BACKUP
    interface ens33   通知选举所用端口(如果本机是eth0,就修改为eth0)
    virtual_router_id 66   虚拟路由的ID号,主备节点的ID号要一致(范围0-255)
    priority 100         主节点的优先级,备节点的优先级必须低 (1-254)
    advert_int 1        VRRP通告间隔时间,1秒
    authentication {
        auth_type PASS    认证机制
        auth_pass 123456  密码,最好使用随机密码(openssl rand -base64 9 生成一个9位数的随机数密码)
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1  添加VIP地址,可以起一个别名,方便观察
    }
}

  配置从节点ka2的keepalived文件 

4、将ka1的配置文件复制到ka2机器上,并对其进行相关的修改。

[root@lvs1keepalived]#scp keepalived.conf 192.168.37.37:/etc/keepalived/
[root@ka2keepalived]#cd /etc/keepalived
[root@ka2keepalived]#vim keepalived.conf 
global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2  改成本机主机名ka2
   #vrrp_mcast_group4 244.100.100.100  此行先注释掉,多播地址有冲突
}

vrrp_instance VI_1 {
    state BACKUP  改为BACKUP
    interface ens33
    virtual_router_id 66
    priority 80  优先级要比主ka1服务器的低
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
}

  5、此时已经配置完ka1和ka2机器,然后启动keepadlive服务

[root@ka1~]#systemctl start keepalived
[root@ka2~]#systemctl start keepalived

  6、此时ka1和ka2都拥有VIP地址。

[root@ka1keepalived]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:99:83:93 brd ff:ff:ff:ff:ff:ff
    inet 192.168.37.27/24 brd 192.168.37.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.37.100/24 scope global secondary ens33:1  添加的VIP地址
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe99:8393/64 scope link 
       valid_lft forever preferred_lft forever

  7、验证效果,在路由器端进行Ping通VIP地址,由于主节点优先级高,此时由主节点ka1承担VIP地址,如果ka1宕机之后,VIP地址就会漂移到备节点上,备用的从节点就会接替位置,还会在工作的过程中。

[root@routernetwork-scripts]#ping 192.168.37.100  路由器端ping VIP地址
PING 192.168.37.100 (192.168.37.100) 56(84) bytes of data.
64 bytes from 192.168.37.100: icmp_seq=1 ttl=64 time=3.53 ms
64 bytes from 192.168.37.100: icmp_seq=2 ttl=64 time=0.411 ms
64 bytes from 192.168.37.100: icmp_seq=3 ttl=64 time=0.413 ms
64 bytes from 192.168.37.100: icmp_seq=4 ttl=64 time=0.413 ms
64 bytes from 192.168.37.100: icmp_seq=5 ttl=64 time=0.640 ms
64 bytes from 192.168.37.100: icmp_seq=15 ttl=64 time=1.23 ms
64 bytes from 192.168.37.100: icmp_seq=16 ttl=64 time=0.362 ms
64 bytes from 192.168.37.100: icmp_seq=17 ttl=64 time=0.813 ms
[root@ka1keepalived]#systemctl stop keepalived  宕机主节点的keepalived服务
[root@routernetwork-scripts]#ping 192.168.37.100  此时的网络不会断掉,只是由从节点进行接管主节点的工作。
PING 192.168.37.100 (192.168.37.100) 56(84) bytes of data.
64 bytes from 192.168.37.100: icmp_seq=1 ttl=64 time=3.53 ms
64 bytes from 192.168.37.100: icmp_seq=2 ttl=64 time=0.411 ms
64 bytes from 192.168.37.100: icmp_seq=3 ttl=64 time=0.413 ms
64 bytes from 192.168.37.100: icmp_seq=4 ttl=64 time=0.413 ms
64 bytes from 192.168.37.100: icmp_seq=5 ttl=64 time=0.640 ms
64 bytes from 192.168.37.100: icmp_seq=15 ttl=64 time=1.23 ms
64 bytes from 192.168.37.100: icmp_seq=16 ttl=64 time=0.362 ms
64 bytes from 192.168.37.100: icmp_seq=17 ttl=64 time=0.813 ms
64 bytes from 192.168.37.100: icmp_seq=18 ttl=64 time=1.07 ms
64 bytes from 192.168.37.100: icmp_seq=19 ttl=64 time=0.406 ms
^C
--- 192.168.37.100 ping statistics ---
19 packets transmitted, 10 received, 1% packet loss, time 18019ms  丢包率是1%
rtt min/avg/max/mdev = 0.362/0.930/3.537/0.916 ms

实验二:实现keepaplived自定义脚本检测功能

nopreempt:定义工作模式为非抢占模式

preempt_delay 300:抢占式模式,节点上线后触发新选举操作的延迟时长,默认模式

定义通知脚本:

notify_master <STRING>|<QUOTED-STRING>:
当前节点成为主节点时触发的脚本
notify_backup <STRING>|<QUOTED-STRING>:
当前节点转为备节点时触发的脚本
notify_fault <STRING>|<QUOTED-STRING>:
当前节点转为“失败”状态时触发的脚本
notify <STRING>|<QUOTED-STRING>:

通用格式的通知触发机制,一个脚本可完成以上三种状态的转换时的通知

1、脚本主要内容:检测到主从发生变化,或错误,给谁发邮件;邮件内容是:在什么时间,谁发生了什么变化

vim /etc/keepalived/notify.sh

#!/bin/bash
#
#********************************************************************
#Author:                liu
#QQ:                    29308620
#Date:                  2019-12-10
#FileName:             notify.sh
#URL:                   http://www.baidu.com
#Description:          The test script
#Copyright (C):         2019 All rights reserved
#********************************************************************
contact='root@localhost'
notify() {
        mailsubject="$(hostname) to be $1, vip floating"
        mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
        echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
        notify master
        ;;
backup)
        notify backup
        ;;
fault)
        notify fault
        ;;
*)
        echo "Usage: $(basename $0) {master|backup|fault}"
        exit 1
        ;;                                                                                                                                       
esac

2、脚本加权限 chmod +x /etc/keepalived/notify.sh  

在keepalived配置文件中调用脚本

脚本的调用方法:

在vrrp_instance VI_1 语句块最后面加下面行

notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"  
notify_fault "/etc/keepalived/notify.sh fault" 

1、修改ka1的keepalived.conf配置文件

global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_mcast_group4 244.100.100.100
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 66
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault" 

   keepalived实现高可用

  2、在ka2机器中添加三条调用脚本的信息。

global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   vrrp_mcast_group4 244.100.100.100
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 66
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master" 
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"         出故障返回此信息                                                                                       
}

    3、先将主从节点的keepalived服务启动:systemctl start keepalived

 4、然后宕机主节点的keepalived服务,此时查看从节点的邮件信息,邮件就会发送到从节点当做主节点的信息。

[root@ka1~]#killall -9 keepalived  关闭掉主节点的keepalived服务

 查看从节点的邮件,此时VIP地址漂移到从节点作为主节点,不会导致失败。

 keepalived实现高可用

   5、当又恢复了主节点的keepalived服务,此时就会查看主节点的邮件,又会恢复到master状态

 keepalived实现高可用

 6、查看从节点的邮件,此时从节点又会返回到BACKUP状态。

 keepalived实现高可用

 打开Wireshark软件,选择VMnet8模式,就会抓取到VIP的地址和密码。

 keepalived实现高可用

实验三:实现双主keepaplived功能

原理:两个keepalived机器都当主节点,当一个机器宕机后,就会将主节点的VIP的IP地址漂移到另一台主节点上,此时另一台主节点就会有两个VIP的IP地址。

配置keepalived1主机

[root@ka1keepalived]#vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id ka1
   #vrrp_mcast_group4 244.110.110.110
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 66
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}


vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 88
    priority 80
advert_int 1
    authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        192.168.37.200/24  dev ens33 label ens33:2  对于另一台192.168.37.200的主机来说,本机就是从服务器
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

配置keepalived2主机

[root@ka2keepalived]#vim /etc/keepalived/keepalived.conf 
global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost 
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   #vrrp_mcast_group4 244.110.110.110
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 66
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 88
    priority 100
    advert_int 1
authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        192.168.37.200/24  dev ens33 label ens33:2  对于另一台192.168.37.200的主机来说,本机是主服务器
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}                                            

效果测试:

宕机掉主节点的IP地址为192.168.37.100的主机,此时该IP地址就会漂移到另一台keepalived服务器上。

keepalived实现高可用

 在另一台keepalived服务器上查看,此时192.168.37.100的地址就飘到此主机上,实现双主配置。

keepalived实现高可用

 总结:当一台主机的机器宕机后,此时主节点的IP地址就会漂移到另一台主机上,此时就实现了双主模式的keepalived服务配置。

实战四:实现keepalived主从方式高可用基于LVS-DR模式的应用实战

   1、在客户端上配置网关和本机的IP地址

[root@centos6 network-scripts]# cat ifcfg-eth0 
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.1.11
GATEWAY=192.168.1.13
PREFIX=24
DNS1=114.114.114.114

 2、在路由器上开启路由转发

[root@router~]#vim /etc/sysctl.conf 开启路由转发功能
net.ipv4.ip_forward=1  
[root@router~]#sysctl -p  生效路由配置文件

    在路由器左接口外网(桥接模式)添加一个IP地址

[root@routernetwork-scripts]#cat ifcfg-ens37
DEVICE=ens37
BOOTPROTO=none
IPADDR=192.168.1.13
PREFIX=24
ONBOOT=yes
DNS1=114.114.114.114

   在路由器右接口内网(NAT模式)添加一个IP地址

[root@routernetwork-scripts]#cat ifcfg-ens33
DEVICE=ens33
BOOTPROTO=none
IPADDR=192.168.37.106
PREFIX=24
ONBOOT=yes
DNS1=114.114.114.114

  4、配置RS1服务器的IP地址,网关指向路由器右边的IP地址

[root@rs1network-scripts]#cat ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.19
PREFIX=24
GATEWAY=192.168.37.106
DNS1=114.114.114.114
ONBOOT=yes
BOOTPROTO=static

 5、配置RS2服务器的IP地址,网关指向路由器有边的IP地址

[root@rs2network-scripts]#cat ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.20
BOOTPROTO=static
GATEWAY=192.168.37.106
PREFIX=24
DNS1=114.114.114.114
ONBOOT=yes

  6、在LVS1服务器上添加一个VIP地址和DIP地址,并将网关指向路由器右端的IP地址

[root@lvsnetwork-scripts]#cat  ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.27    DIP地址
PREFIX=24
BOOTPROTO=static
GATEWAY=192.168.37.106   指向路由器右端IP地址(192.168.37.106)的网关
ONBOOT=yes
DNS1=114.114.114.114

  7、在LVS2服务器上添加一个VIP地址和DIP地址,并将网关指向路由器右端的IP地址

[root@lvsnetwork-scripts]#cat  ifcfg-ens33
DEVICE=ens33
IPADDR=192.168.37.37    DIP地址
PREFIX=24
BOOTPROTO=static
GATEWAY=192.168.37.106   指向路由器右端IP地址(192.168.37.106)的网关
ONBOOT=yes
DNS1=114.114.114.114

在lvs-server-master 主节点配置文件

[root@ka1]#vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id ka1
   #vrrp_mcast_group4 244.110.110.110
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 66
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

virtual_server 192.168.37.100 80 {
    delay_loop 6   服务轮询时间间隔
    lb_algo rr    rr轮询算法
    lb_kind DR    LVS集群模式:RD模式
    protocol TCP
    persistence_timeout 600   持久连接600s,一直访问一台主机
    sorry_server 127.0.0.7 80  当主机宕机时提示道歉信息

    real_server 192.168.37.19 80 {  RS1服务器的信息
        weight 1
        HTTP_GET {
            url {
              path /    返回到根目录
              status_code 200
            }
            connect_timeout 3  连接超时时长
            nb_get_retry 3     重建次数
            delay_before_retry 3 重建间隔
        }
    }
    real_server 192.168.37.20 80 {  RS2服务器信息
        weight 1
        HTTP_GET {
            url {
              path /testurl/test.jsp
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
} 

启动keepalived服务,并查看ipvsadm集群规则。

[root@ka1keepalived]#systemctl restart keepalived
[root@ka1keepalived]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.37.100:80 rr
-> 192.168.37.19:80 Route 1 0 4   此时相关的RS1服务策略已配置
-> 192.168.37.20:80 Route 1 0 4   此时相关的RS2服务策略已配置

在lvs-server-master 从节点配置文件

[root@ka2]#vim /etc/keepalived/keepalived.conf 
global_defs {
   notification_email {
       root@localhost
   }
   notification_email_from  keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   #vrrp_mcast_group4 244.110.110.110
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 66
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100/24  dev ens33 label ens33:1
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

virtual_server 192.168.37.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    persistence_timeout 600  持久连接,一直连接一台主机600s
    sorry_server 127.0.0.7 80

real_server 192.168.37.19 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.37.20 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

在real server1 修改配置文件

[root@rs1~]#vim lvs_dr_rs.sh 
#!/bin/bash
#Author:liu
#Date:2017-08-13
vip=192.168.37.100       指向VIP地址                                                                                                                         
mask='255.255.255.255'
dev=lo:1
rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
#service httpd start &> /dev/null && echo "The httpd Server is Ready!"
#echo "<h1>`hostname`</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

执行脚本:

[root@rs1~]#bash lvs_dr_rs.sh  start
The RS Server is Ready!

永久生效,可以写到配置文件中

vim /etc/sysctl.conf

net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

sysctl -p  使配置文件生效。

sysctl -p 

在real server2修改配置文件

[root@rs2~]#vim lvs_dr_rs.sh 
#!/bin/bash
#Author:liu
#Date:2017-08-13
vip=192.168.37.100                                                                                                                               
mask='255.255.255.255'
dev=lo:1
rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
#service httpd start &> /dev/null && echo "The httpd Server is Ready!"
#echo "<h1>`hostname`</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

执行脚本:

[root@rs2~]#bash lvs_dr_rs.sh  start 
The RS Server is Ready!

添加RS1和RS2服务器的页面文件,并启动httpd服务

[root@rs1~]#echo 192.168.37.19 RS1 > /var/www/html/index.html
[root@rs1~]#systemctl start httpd
[root@rs2~]#echo 192.168.37.20 RS2 > /var/www/html/index.html
[root@rs2~]#systemctl start httpd

客户端测试效果:

此时在客户端已经访问到RS1和RS2的httpd服务下的网页

[root@router~]#while : ;do curl 192.168.37.100 ;sleep 0.5;done
192.168.37.20 RS2
192.168.37.19 RS1
192.168.37.20 RS2
192.168.37.19 RS1

此时停掉LVS1的keepalived服务

[root@ka1keepalived]#systemctl stop keepalived.service 

客户端仍然还在访问过程,不会被断开。

  keepalived实现高可用

此时LVS2上有ipvsadm规则,并将LVS1服务的VIP地址漂移到LVS2服务器上,解决了LVS1故障问题。

   keepalived实现高可用

  当有启动LVS1机器的keepalived服务,由于LVS1的优先级高,ipvsadm服务规则和VIP地址就又飘到了主服务器上,LVS2的服务器上也有ipvsadm规则和VIP地址。

 keepalived实现高可用

  验证两个RS服务器都宕机效果:

1、在LVS1和LVS2服务器上分别添加网页显示页面。

[root@ka1~]#echo sorry server > /var/www/html/index.html  在keepalived1写入道歉信息
[root@ka1~]#systemctl start httpd
[root@ka2~]#echo sorry server1 > /var/www/html/index.html 在keepalived2写入道歉信息
[root@ka2~]#systemctl start httpd

 2、此时宕机RS1和RS2及LVS1服务器,在客户端上显示sorry  server1.

[root@rs1~]#systemctl stop httpd  宕机RS1服务器
[root@rs2~]#systemctl stop httpd  宕机RS2服务器
[root@ka1~]#systemctl stop keepalived 宕机keepalived1服务器

 在客户端访问VIP返回的结果就是keepalived2服务器道歉的信息。

 keepalived实现高可用

 

 

 

 

 

 

  

  

 

 

  

 

 

 

  

 

 

 

 

  

 

  

  

 

 

  

  

  

  

  

 

 

  

 

 

 

上一篇:Linux中双网卡的配置


下一篇:python基础之二:占位符、格式化输出、while else 、逻辑运算