用HAproxy+keepalived+mysql Replication 构建基于企业级负载均衡

 最近用HAproxy+keepalived+mysql复制测试高可用性Linux系统集群。
   HAProxy是一款免费的提供高可用性、负载均衡以及基于TCP(第四层)和HTTP(第七层)应用的代理软件,借助HAProxy可以快速并且可靠的提供基于TCP和HTTP应用的代理解决方案。
   Keepalived主要作用是LoadBalance master和LoadBalance backup之间的健康检查,实现故障转换。
   Mysql Replication主要作用是提高mysql并处理数据的能力以及实现容灾备份的作用。

项目拓扑图:

用HAproxy+keepalived+mysql Replication 构建基于企业级负载均衡

Haproxy服务器及其IP地址规划:

用HAproxy+keepalived+mysql Replication 构建基于企业级负载均衡

项目实施:

1.Mysql主从复制配置。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
1)在master服务器上创建mysql用户(授权复制账户)。
grant replication slave on *.* to'rep'@'192.168.1.244'identified by 'rep123';
2)编辑master服务器的mysql配置文件my.cnf。
server-id = 1 //指定服务器的ID
log-bin = mysql-bin //开启二进制日志
binlog-ignore = mysql //忽略mysql和information_schema 数据库
binlog-ignore = information_schema
binlog-do-db = blog    //同步数据库,默认同步所有数据库
3)查看master状态。
mysql> show master status;
+---------------+----------+--------------+--------------------------+
| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB         |
+---------------+----------+--------------+--------------------------+
| binlog.022343 |   339244 | blog         | mysql,information_schema |
+---------------+----------+--------------+--------------------------+
1 row in set (0.00 sec)
4)在slave端创建数据库blog,导出master端的blog库,导入到此库,并修改mysql主配置文件my.cnf server-id = 2 重启mysql数据库。
mysql> change master to
    -> master_host='192.168.1.243',
    -> master_port=3306,
    -> master_user='rep',
    -> master_password='rep123',
    -> master_log_file='binlog.022343',
    -> master_log_pos=339244;
5)mysql主从同步测试,show slave status\G;能看到Slave_IO_Running和Slave_SQL_Running都为YES即可。
mysql> show slave status\G;
*************************** 1. row ***************************
             Slave_IO_State: Waiting for master to send event
                Master_Host: 192.168.1.243
                Master_User: rep
                Master_Port: 3306
              Connect_Retry: 60
            Master_Log_File: binlog.022343
        Read_Master_Log_Pos: 339110
             Relay_Log_File: relaylog.005481
              Relay_Log_Pos: 339244
      Relay_Master_Log_File: binlog.022343
           Slave_IO_Running: Yes
          Slave_SQL_Running: Yes
            Replicate_Do_DB:
        Replicate_Ignore_DB: mysql,test,information_schema
         Replicate_Do_Table:
     Replicate_Ignore_Table:
    Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
                 Last_Errno: 0
                 Last_Error:
               Skip_Counter: 0
        Exec_Master_Log_Pos: 339110
            Relay_Log_Space: 339244
            Until_Condition: None
             Until_Log_File:
              Until_Log_Pos: 0
         Master_SSL_Allowed: No
         Master_SSL_CA_File:
         Master_SSL_CA_Path:
            Master_SSL_Cert:
          Master_SSL_Cipher:
             Master_SSL_Key:
      Seconds_Behind_Master: 0
1 row in set (0.00 sec)

2.Haproxy安装及其配置,master和backup安装配置都是完全一样的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gz
tar -zxvf haproxy-1.4.24.tar.gz
cd haproxy-1.4.24
make TARGET=linux26 PREFIX=/usr/local/haproxy
make install PREFIX=/usr/local/haproxy
cd /usr/local/haproxy
mkdir conf logs   //在此目录下面建立conf,logs目录分别存放HAproxy的配置文件,PID文件和日志文件。
vim conf/haproxy.conf
global
        maxconn 50000
        chroot /usr/local/haproxy
        uid 99
        gid 99
        daemon
        quiet
        nbproc  2
        pidfile /usr/local/haproxy/logs/haproxy.pid
        #debug
defaults
        log     global
        mode    http
        option  httplog   #每次请求完毕后主动关闭http通道
        option  dontlognull #不记录健康检查的日志信息
        option forwardfor
        option redispatch
        option abortonclose
        retries 3
        log 127.0.0.1 local3
        maxconn 20000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000
listen 192.168.1.236
       bind *:80
       mode http
       stats uri /admin                    #后端服务器状态查看地址
       stats auth admin:admin             #状态查看页面登陆帐号密码
       balance source   #调度算法,source是和nginx的ip_hash同理,解决session问题
       option httpclose
       option forwardfor
server web1 192.168.1.248:80 weight 5 check inter 2000 rise 2 fall 5
server web2 192.168.1.249:8080 weight 5 check inter 2000 rise 2 fall 5
#启动Haproxy服务
/usr/local/haproxy/sbin/haproxy -f haproxy.conf
#haproxy启动脚本
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="$@"
start()
{
echo "START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy -f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo "STOP HAPoxy Listen"
kill -TTOU $(cat $BASE_DIR/logs/haproxy.pid)
echo "STOP HAPoxy process"
kill -USR1 $(cat $BASE_DIR/logs/haproxy.pid)
}
case $ARGV in
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo "hactl.sh [start|restart|stop]"
esac
exit $ERROR

3.Haproxy开启系统日志支持。

1
2
3
4
5
6
7
8
9
vim /etc/syslog.conf
#添加:
local3.*        /var/log/haproxy.log
local0.*        /var/log/haproxy.log
vim /etc/sysconfig/syslog
#修改:
SYSLOGD_OPTIONS="-r -m 0"
#重新启动syslog服务
/etc/init.d/syslog restart

4.Keepalived的安装配置。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
wget http://www.keepalived.org/software/keepalived-1.2.5.tar.gz
ln -s /usr/src/kernels/2.6.18-164.el5-x86_64/ /usr/src/linux
tar -zxvf keepalived-1.2.5.tar.gz
cd keepalived-1.2.5
./configure --prefix=/usr/local/keepalived
make && make install
cp keepalived/etc/init.d/keepalived.rh.init /etc/init.d/keepalived
chmod +x /etc/init.d/keepalived
cp keepalived/etc/init.d/keepalived.sysconfig /etc/sysconfig/keepalived
chkconfig --add keepalived
chkconfig --level 35 keepalived on
cp /usr/local/keepalived/sbin/keepalived /bin/
mkdir -p /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
#Haproxy master配置文件
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
    shifeng_zhang88@163.com
    }
    notification_email_from shifeng_zhang88@163.com
    smtp_server smtp.163.com
    smtp_connect_timeout 30
    router_id LVS_Master
}
vrrp_script chk_http_port {
    script  "/etc/keepalived/check_haproxy.sh"
    interval        5       #脚本执行间隔
    weight         -5       #执行脚本后优先级变更:5表示优先级+5;-5则表示优先级-5
}
vrrp_instance VI_A {
    state MASTER
    interface eth0
    virtual_router_id 50
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass sfzhang1109
    }
track_script {
    chk_http_port
    }
    virtual_ipaddress {
        192.168.1.236    #haproxy虚拟IP
    }
}
#Haproxy backup配置文件只需改变state和priority的值
state BACKUP
priority 50
#启动keepalived服务
/etc/init.d/keepalived start

5.编辑check_haproxy.sh脚本,需要将haproxy启动脚本放到/etc/init.d/里面。

1
2
3
4
5
6
7
8
9
10
11
12
13
vim /etc/keepalived/check_haproxy.sh
#!/bin/bash
A=`ps -C haproxy --no-header |wc -l`
if [ $A -eq 0 ];then
/etc/init.d/haproxy restart
echo "Start haproxy" &> /dev/null
sleep 3
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
/etc/init.d/keepalived stop
echo "Stop keepalived" &> /dev/null
fi
fi
#chomd +x /etc/keepalived/check_haproxy.sh

6.Haproxy+Keepalived高可用性测试。

1)Haproxy+Keepalived 高可用测试。

   测试方法:停止master的keepalived服务,查看备用的keepalived的日志发现当主节点宕机时,备用节点角色会立即变为主节点,启用VIPS协议,并把VIP地址立刻绑定到eth0网卡上面,当主节点恢复时则做相反的工作。

用HAproxy+keepalived+mysql Replication 构建基于企业级负载均衡

2)Haproxy+Keepalived负载均衡测试

   测试方法:当客户端访问VIP的时候,haproxy根据设置的调度算法和权重把访问请求分发到后端的WEB服务器上面,从而实现了负载均衡的功能。

3)Haproxy+Keepalived 故障转移测试

   测试方法:当后端的WEB服务器down机时,haproxy会自动检测到并把请求发送到正常的服务器上面,通过haproxy监控页面(http://192.168.1.235/admin)可以查看。

用HAproxy+keepalived+mysql Replication 构建基于企业级负载均衡

    说明:(希望大家提出宝贵的意见)

      1)拓扑图只是项目的一个雏形,后期还要添加监控服务器和邮件等服务器。

      2)Mysql主库负责用户数据的写入,slave负责用户数据的读取和数据的备份等,除了要监控mysql主从外还要监控Seconds_Behind_Master选项,以免产生主从数据不同步或者延时。

      3)当WEB的负载很大的时候,可以在haproxy添加多台物理机即可,并考虑把项目的图片分离出来做成单独的图片服务器。










本文转自 sfzhang 51CTO博客,原文链接:http://blog.51cto.com/sfzhang88/1229620,如需转载请自行联系原作者

上一篇:全球排名第一的宽表数据库免费开营了!


下一篇:【0601 - 0605 直播导视 | PPT 下载】阿里巴巴内部研发效能峰会首次对外直播预热来啦!