架构如上,但是其实keepalived、haproxy、Mycat都可以多台(比如keepalived、haproxy、Mycat各3台,3台keepalived抢占vip,然后抢到vip的haproxy提供服务,每台haproxy后面挂3个mycat)
如下是我根据自己的开发环境,自己安装的步骤:
keepalived-2.0.6.tar.gz
haproxy-1.8.16.tar.gz
Mycat-server-1.6.6.1-release-20181031195535-linux.tar.gz
主机1:ip192.168.0.200 安装keepalived、haproxy、Mycat
主机2:ip192.168.0.199 安装keepalived、haproxy、Mycat
vip:192.168.0.206
keepalived与haproxy需安装在同一台机器上
运作理解:主机1与主机2上的keepalived会为该服务器抢占vip,抢占到vip后,对该主机的访问可以通过原来的ip(ip192.168.0.200或者192.168.0.199)访问,也可以直接通过vip(192.168.0.206)访问,当其中一台宕机时,vip会漂移到另一台机器上,当我们访问192.168.0.206时haproxy会根据配置将请求分发到两台Mycat上实现负载均衡,并且haproxy会检测mycat是否存活。
一、haproxy安装
1、安装haproxy(200和199都安装,安装方式一样)
下载地址:
https://src.fedoraproject.org/repo/pkgs/haproxy/
国内镜像地址:http://pkgs.fedoraproject.org/repo/pkgs/haproxy/
查看内核版本
uname -r
[root@localhost haproxy]# uname -r
3.10.0-957.5.1.el7.x86_64
根据内核版本选择编译参数:
这里内核是3.10的可以选择TARGET=linux310 或者 选择 linux2628,安装haproxy
tar -zxvf haproxy-1.8.16.tar.gz
cd haproxy-1.8.16
make TARGET=linux310 ARCH=x86_64 PREFIX=/usr/local/haproxy
make install PREFIX=/usr/local/haproxy
cd /usr/local/haproxy
安装成功后,查看版本
[root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -v
HA-Proxy version 1.8.16-5c3f237 2018/12/21
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>
2、配置haproxy
cd /usr/local/haproxy
touch haproxy.cfg
vim /usr/local/haproxy/haproxy.cfg
写入一下内容:
global
#设置日志
log 127.0.0.1 local0
chroot /usr/local/haproxy
#用户与用户组
user root
group root
#定义每个haproxy进程的最大连接数 ,由于每个连接包括一个客户端和一个服务器端,所以单个进程的TCP会话最大数目将是该值的两倍。
maxconn 4096
# 以守护进程的方式运行
daemon
defaults
log global
#日志中不记录空连接
option dontlognull
# 定义连接后端服务器的失败重连次数,连接失败次数超过此值后将会将对应后端服务器标记为不可用
retries 3
option redispatch
# 设置成功连接到一台服务器的最长等待时间,默认单位是毫秒
timeout connect 5000
# 设置连接客户端发送数据时的成功连接最长等待时间,默认单位是毫秒
timeout client 50000
# 设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒
timeout server 60000
#统计页面
listen admin_stats
bind 192.168.0.206:48800
mode http
#采用http日志格式
option httplog
#统计页面自动刷新时间
stats refresh 30s
#统计页面url
stats uri /admin_stats
#统计页面密码框上提示文本
stats realm Haproxy Manager
#统计页面用户名和密码设置
stats auth admin:admin
#隐藏统计页面上HAProxy的版本信息
stats hide-version
listen mycat_service
# 绑定192.168.0.206:8067端口访问mycat8066端口
bind 192.168.0.206:8067
# 定义为tcp模式
mode tcp
#采用http日志格式
option tcplog
# 开启对后端服务器的健康检测
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
# 设置haproxy的调度算法
balance roundrobin
#根据调度分配到真实的后台地址,参数解释:port 48700:检测端口48700, inter 5s:5秒检测一次,rise 2:检测成功2次表示服务器可用,fall 3:检测失败3次后表示服务器不可用
server mycat_200 192.168.0.200:8066 check port 48700 inter 5s rise 2 fall 3
server mycat_199 192.168.0.199:8066 check port 48700 inter 5s rise 2 fall 3
#设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒
timeout server 60000
listen mycat_admin
#绑定192.168.0.206:9067端口访问mycat9066端口
bind 192.168.0.206:9067
mode tcp
option tcplog
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
balance roundrobin
server mycat_200 192.168.0.200:9066 cookie 1 check port 48700 inter 5s rise 2 fall 3
server mycat_199 192.168.0.199:9066 cookie 2 check port 48700 inter 5s rise 2 fall 3
timeout server 60000
上面的配置中 timeout server 可以配置大一点,不然经常会出现有时候闪断连接mysql的情况
启动和停止haproxy命令
/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg #启动
killall haproxy #停止
[root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg
[WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_200' as HTTP mode is disabled.
[WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_199' as HTTP mode is disabled.
[ALERT] 170/114135 (88204) : Starting proxy admin_stats: cannot bind socket [192.168.0.206:48800]
[ALERT] 170/114135 (88204) : Starting proxy mycat_service: cannot bind socket [192.168.0.206:8067]
[ALERT] 170/114135 (88204) : Starting proxy mycat_admin: cannot bind socket [192.168.0.206:9067]
3、配置haproxy记录日志功能
默认haproxy是不记录日志的,为了记录日志还需要配置syslog模块,在linux下是rsyslogd服务,需要先安装rsyslog
yum install rsyslog -y
[root@localhost haproxy]# yum install rsyslog -y
Loaded plugins: fastestmirror, langpacks
Determining fastest mirrors
* base: mirrors.163.com
* extras: mirrors.163.com
* updates: mirrors.cn99.com
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
packages-microsoft-com-prod | 2.9 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/3): extras/7/x86_64/primary_db | 205 kB 00:00:00
(2/3): packages-microsoft-com-prod/primary_db | 170 kB 00:00:00
(3/3): updates/7/x86_64/primary_db | 6.4 MB 00:00:01
Package rsyslog-8.24.0-34.el7.x86_64 already installed and latest version
Nothing to do
如果没有目录
/etc/rsyslog.d 就执行
mkdir /etc/rsyslog.d
cd /etc/rsyslog.d/
touch haproxy.conf
vim haproxy.conf
写入一下内容
$ModLoad imudp
$UDPServerRun 514
local0.* /var/log/haproxy.log
vim /etc/rsyslog.conf
修改两处:1在#### RULES ####上面一行加入以下内容
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
第二处在local7.* /var/log/boot.log下面增加
local0.* /var/log/haproxy.log
重启rsyslog服务
systemctl restart rsyslog.service
将rsyslog加入自动启动服务
systemctl enable rsyslog.service
二、配置监听mycat是否存活
在Mycat server1 Mycat server2上都需要添加检测端口48700的脚本,为此需要用到xinetd,xinetd为linux系统的基础服务,在xinetd目录下面增加脚本与端口的映射配置文件。
yum install xinetd -y
cd /etc/xinetd.d
touch mycat_status
vim /etc/xinetd.d/mycat_status
写入以下内容
service mycat_status
{
flags = REUSE
socket_type = stream
port = 48700
wait = no
user = root
server =/usr/local/bin/mycat_status
log_on_failure += USERID
disable = no
}
创建xinetd启动服务脚本
vim /usr/local/bin/mycat_status
#!/bin/bash
#/usr/local/bin/mycat_status.sh
# This script checks if a mycat server is healthy running on localhost. It will
# return:
#
# "HTTP/1.x 200 OK\r" (if mycat is running smoothly)
#
# "HTTP/1.x 503 Internal Server Error\r" (else)
mycat=`/root/mycat/bin/mycat status |grep 'not running'| wc -l`
if [ "$mycat" = "0" ];
then
/bin/echo -e "HTTP/1.1 200 OK\r\n"
else
/bin/echo -e "HTTP/1.1 503 Service Unavailable\r\n"
fi
修改脚本文件权限
chmod 777 /usr/local/bin/mycat_status
chmod 777 /etc/xinetd.d/mycat_status
将启动脚本加入服务
vim /etc/services
在末尾加入
mycat_status 48700/tcp # mycat_status
重启xinetd服务
systemctl restart xinetd.service
验证mycat_status服务是否启动成功
netstat -antup|grep 48700
注意:/etc/xinetd.d/mycat_status里的service mycat_status(服务名)和port = 48700(端口)要与 /etc/services里的相同
三、keepalived安装
1、安装依赖
yum install -y openssl-devel openssl libnl libnl-devel libnfnetlink-devel
2、keepalived安装
(安装keepalived的时候经常keepalived启动不了,或者说启动了但是haproxy切不了,后来试了keepalived2.0.6 2.0.16 1.4.5 都还是一样,最后按照mycat权威指南换成1.2.13还是不行,后来终于发现可能是
我改了权威指南上写的/etc/keepalived/scripts/check_haproxy.sh 把两行的
$STARTHAPROXY 改成了第二行是$STOPKEEPALIVED 导致每次keepalived一启动就被停了,后面我把两行都写成
$STARTHAPROXY就没问题了,出问题的时候还可以多查看 /usr/local/keepalived/var/log/keepalived-haproxy-state.log 和/var/log/messages 看看是否check或者其他什么的。 还有就是下面这一段我试试了去掉后手动打字也有影响,但是可能也没影响,
后续再安装一次验证一下
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
insterval 2
weight 2
}
)
cd /root
wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz (如果wget下载不了可以去keepalived官网下载 https://www.keepalived.org/download.html)
tar -zxvf keepalived-1.2.13.tar.gz
cd keepalived-1.2.13
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ (这里不同的keepalived版本路径可能不同)
mkdir /etc/keepalived
cd /etc/keepalived/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived
mkdir -p /usr/local/keepalived/var/log
2、keepalived配置
a、查看网卡
ifconfig
我这里网卡是ens33所以配置里的interface为ens33,主机1与主机2配置基本一样,state与priority 不同。b、先配置主机2 192.168.0.199的keepalived
vim /etc/keepalived/keepalived.conf
! Configuration Fileforkeepalived
global_defs {
router_id ip199
}
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
insterval 2
weight 2
}
#vrrp实例
vrrp_instance VI_1 {
state BACKUP
#本机网卡
interface ens33
##VRRP组名,两个节点的设置必须一样,以指明各个节点属于同一VRRP组
virtual_router_id 51
#优先级,数值愈大,优先级越高
priority 120
#检查间隔,默认1秒
advert_int 1
#设置认证
authentication {
#认证方式
auth_type PASS
#认证密码
auth_pass 1111
}
#虚拟IP地址池,可以有多个IP,每个IP占一行
virtual_ipaddress {
192.168.0.206 dev ens33 scope global
}
track_script {
#调用脚本check_haproxy.sh检查haproxy是否存活
chkhaproxy
}
#指定当切换到master时,执行的脚本
notify_master /etc/keepalived/scripts/haproxy_master.sh
#指定当切换到backup时,执行的脚本
notify_backup /etc/keepalived/scripts/haproxy_backup.sh
#故障时执行的脚本
notify_fault /etc/keepalived/scripts/haproxy_fault.sh
notify_stop /etc/keepalived/scripts/haproxy_stop.sh
}
c、配置主机1 192.168.0.200的keepalived
! Configuration Fileforkeepalived
global_defs {
router_id ip200
}
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.206 dev ens33 scope global
}
track_script {
chkhaproxy
}
notify_master /etc/keepalived/scripts/haproxy_master.sh
notify_backup /etc/keepalived/scripts/haproxy_backup.sh
notify_fault /etc/keepalived/scripts/haproxy_fault.sh
notify_stop /etc/keepalived/scripts/haproxy_stop.sh
}
3、创建配置文件和脚本
mkdir /etc/keepalived/scripts
a、创建检查haproxy是否存活的脚本
vim /etc/keepalived/scripts/check_haproxy.sh
#!/bin/bash
STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
STOPKEEPALIVED="/etc/init.d/keepalived stop"
LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
echo "[check_haproxy status]" >> $LOGFILE
A=`ps -C haproxy --no-header |wc -l`
echo "[check_haproxy status]" >> $LOGFILE
date >> $LOGFILE
#如果没有haproxy进程存在,就启动haproxy,停止keepalived,并写入日志
if [ $A -eq 0 ];then
echo $STARTHAPROXY >> $LOGFILE
$STARTHAPROXY >> $LOGFILE 2>&1
sleep 5
fi
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
exit 0
else
exit 1
fi
b、创建切换到MASTER与BACKUP脚本
vim /etc/keepalived/scripts/haproxy_master.sh
vim /etc/keepalived/scripts/haproxy_backup.sh
在两个文件中写入以下内容
#!/bin/bash
STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
STOPHAPROXY=`ps -ef | grep sbin/haproxy | grep -v grep | awk '{print $2}'| xargs kill -s 9`
LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "stop haproxy...." >> $LOGFILE 2>&1
$STOPHAPROXY >> $LOGFILE 2>&1
echo "start haproxy...." >> $LOGFILE 2>&1
$STARTHAPROXY >> $LOGFILE 2>&1
echo "haproxy stared ..." >> $LOGFILE
c、创建故障时执行的脚本
vim /etc/keepalived/scripts/haproxy_fault.sh
#!/bin/bash
LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE
d、创建停止vrrp脚本
vim /etc/keepalived/scripts/haproxy_stop.sh
#!/bin/bash
LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE
4、赋予脚本可执行权限
chmod 777 /etc/keepalived/scripts/*
5、启动keepalived
systemctl start keepalived.service
6、加入开机启动
systemctl enable keepalived.service
四、测试
在浏览器输入http://192.168.0.206:48800/admin_stats
查看8067与9067端口是否在监听
连接8067端口
mysql -uroot -p123456 -h192.168.0.206 -P8067
可以连接,到此就搭建完成了