RocketMQ 异步复制集群部署安装

防伪码:他日若遂凌云志,敢笑黄巢不丈夫。


1.RacketMQ多种部署模式

RocketMQ Broker集群部署方式主要有以下几种:(Slave 不可写,但可读)
1.1.单个Master
这种方式风险较大,一旦Broker 重启或者宕机时,会导致整个服务不可用,不建议线上环境使用。
1.2.多Master模式
一个集群无 Slave,全是 Master,例如 2 个 Master 或者 3 个 Master。

优点:配置简单,单个Master 宕机或重启维护对应用无影响,在磁盘配置为 RAID10 时,即使机器宕机不可恢复情况下,由与 RAID10 磁盘非常可靠,消息也不会丢(异步刷盘丢失少量消息,同步刷盘一条不丢)。性能最高。

缺点:单台机器宕机期间,这台机器上未被消费的消息在机器恢复之前不可订阅,消息实时性会受到受到影响。

# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 & 
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-noslave/broker-a.properties >/dev/null 2>&1 & 
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-noslave/broker-b.properties >/dev/null 2>&1 & 
1.3.多Master多Slave模式,异步复制
每个 Master 配置一个 Slave,有多对Master-Slave,HA 采用异步复制方式,主备有短暂消息延迟,毫秒级。

优点:即使磁盘损坏,消息丢失的非常少,且消息实时性不会受影响,因为 Master 宕机后,消费者仍然可以从 Slave 消费,此过程对应用透明。不需要人工干预。性能同多 Master 模式几乎一样。

 缺点:Master 宕机,磁盘损坏情况,会丢失少量消息。

# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 & 
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-a.properties >/dev/null 2>&1 & 
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-b.properties >/dev/null 2>&1 & 
#在机器 C,启动第一个 Slave
nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-a-s.properties >/dev/null 2>&1 & 
# 在机器 D,启动第二个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-b-s.properties >/dev/null 2>&1 & 
1.4.多Master多Slave模式,同步双写
每个 Master 配置一个 Slave,有多对Master-Slave,HA 采用同步双写方式,主备都写成功,向应用返回成功。

优点:数据与服务都无单点,Master宕机情况下,消息无延迟,服务可用性与数据可用性都非常高

缺点:性能比异步复制模式略低,大约低 10%左右,发送单个消息的 RT 会略高。目前主宕机后,备机不能自动切换为主机,后续会支持自动切换功能。

# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 & 
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-a.properties >/dev/null 2>&1 & 
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-b.properties >/dev/null 2>&1 & 
# 在机器 C,启动第一个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-a-s.properties >/dev/null 2>&1 & 
# 在机器 D,启动第二个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-b-s.properties >/dev/null 2>&1 & 

注:
以上 Broker 与 Slave 配对是通过指定相同的brokerName 参数来配对,Master 的 BrokerId 必须是 0,Slave 的BrokerId 必须是大与 0 的数。另外一个 Master 下面可以挂载多个 Slave,同一 Master 下的多个 Slave 通过指定不同的 BrokerId 来区分。
除此之外,nameserver也需要集群。我们的环境采用多nameserver多master多slave异步复制模式

2.环境准备

RocketMQ 异步复制集群部署安装

角色
主机名 IP port OS 配置
Nameserver1 INVOICE-HX-01 10.162.23.70 9876 Redhat7.2 16C32G100G
Nameserver2 INVOICE-HX-02 10.162.23.71 9876 Redhat7.2 16C32G100G
Master1 INVOICE-HX-01 10.162.23.70 10911 10922 Redhat7.2 16C32G100G
Master2 INVOICE-HX-02 10.162.23.71 10911 10922 Redhat7.2 16C32G100G
Slave1 INVOICE-HX-03 10.162.23.72 10911 10922 Redhat7.2 16C32G100G
Slave2 INVOICE-HX-04 10.162.23.73 10911 10922 Redhat7.2 16C32G100G

moosefs安装前准备:
# RocketMQ软件包:alibaba-rocketmq-3.2.6.tar.gz
# JDK 1.7.79
# SELINUX :disable
# 安装路径:/opt/server/rocketmq
# 日志路径:/opt/var/logs/rocketmq

# 所有主机配置hosts
# cat /etc/hosts
10.162.23.70 INVOICE-HX-01
10.162.23.71 INVOICE-HX-02
10.162.23.72 INVOICE-HX-03
10.162.23.72 INVOICE-HX-04

注:hosts解析必须是真实的hostname

3.RocketMQ多M多异步复制集群安装

3.1.JDK安装
# 所有角色上安装
# 上传jdk-7u79-linux-x64.gz
# mkdir /opt/server -p
# tar zvxf jdk-7u79-linux-x64.gz -C /opt/server

# 修改 /etc/profile
# vi /etc/profile
# jdk
JAVA_HOME=/opt/server/jdk1.7.0_79
CLASSPATH=$JAVA_HOME/jre/lib/rt.jar:.
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH CLASSPATH

# source /etc/profile
# 检查java版本:
# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

3.2.RocketMQ 解压安装
# 所有角色上安装
# 上传alibaba-rocketmq-3.2.6.tar.gz
# tar zxvf alibaba-rocketmq-3.2.6.tar.gz
# mv alibaba-rocketmq  /opt/server/rocketmq
#安装包bin下文件没有执行权限
# chmod +x /opt/server/rocketmq/bin/*

# 创建数据存储目录
# mkdir -p /opt/data/rocketmq/data/store/commitlog
# 创建日志目录
# mkdir -p /opt/var/logs/rocketmq/
# 修改日志目录
# cd /opt/server/rocketmq/conf
# sed -i  's#${user.home}/logs#/opt/var/logs/rocketmq#g' *.xml
# 查看修改
# grep logs *.xml
3.3.RocketMQ Nameserver启动
# nohup /opt/server/rocketmq/bin/mqnamesrv &
# 验证
# cat nohup.out
The Name Server boot success.
# netstat -anlp |grep 9876
tcp        0      0 :::9876                     :::*                        LISTEN      23750/java

# 开机启动
# cat /etc/rc.local
# RocketMQ nameserver
nohup /opt/server/rocketmq/bin/mqnamesrv &
3.4.RocketMQ Master1 Slave1
# 修改master1(10.162.23.70)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=0
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88

storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000

checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

# 启动master1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
# netstat -anlp|grep java        
tcp        0      0 :::10911                    :::*                        LISTEN      24220/java         
tcp        0      0 :::10912                    :::*                        LISTEN      24220/java

# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ master-a
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &

# 修改slave1(10.162.23.72)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=1
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88

storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000

checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

# 启动salve1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &

# netstat -anlp|grep java
tcp        0      0 :::10911                    :::*                        LISTEN      23774/java         
tcp        0      0 :::10912                    :::*                        LISTEN      23774/java

# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ slave-a
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &

3.5.RocketMQ Master2 Slave2
# 修改master2(10.162.23.71)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=0
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88

storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000

checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

# 启动master2
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

# 验证
# cat nohup.out
The Name Server boot success.
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-a, 10.162.23.71:10911] boot success.
# netstat -anlp|grep java        
tcp        0      0 :::10911                    :::*                        LISTEN      24220/java         
tcp        0      0 :::10912                    :::*                        LISTEN      24220/java

# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ master-b
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

# 修改slave2(10.162.23.73)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=1
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88

storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000

checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

# 启动salve1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

# 验证
# cat nohup.out
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-b, 10.162.23.73:10911] boot success. and name server is 10.162.23.70:9876;10.162.23.71:9876
# netstat -anlp|grep java
tcp        0      0 :::10911                    :::*                        LISTEN      23774/java         
tcp        0      0 :::10912                    :::*                        LISTEN      23774/java

# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ slave-b
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

注:服务停止直接采用kill java 进程的方法

3.6.RocketMQ nameserver 接口
10.162.23.70:9876;10.162.23.71:9876


上一篇:Error: <spyOn> : handleError() method does not exist


下一篇:maven引入外部jar包