防伪码:他日若遂凌云志,敢笑黄巢不丈夫。
1.RacketMQ多种部署模式
RocketMQ Broker集群部署方式主要有以下几种:(Slave 不可写,但可读)
1.1.单个Master
这种方式风险较大,一旦Broker 重启或者宕机时,会导致整个服务不可用,不建议线上环境使用。
1.2.多Master模式
一个集群无 Slave,全是 Master,例如 2 个 Master 或者 3 个 Master。
优点:配置简单,单个Master 宕机或重启维护对应用无影响,在磁盘配置为 RAID10 时,即使机器宕机不可恢复情况下,由与 RAID10 磁盘非常可靠,消息也不会丢(异步刷盘丢失少量消息,同步刷盘一条不丢)。性能最高。
缺点:单台机器宕机期间,这台机器上未被消费的消息在机器恢复之前不可订阅,消息实时性会受到受到影响。
# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 &
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-noslave/broker-a.properties >/dev/null 2>&1 &
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-noslave/broker-b.properties >/dev/null 2>&1 &
1.3.多Master多Slave模式,异步复制
每个 Master 配置一个 Slave,有多对Master-Slave,HA 采用异步复制方式,主备有短暂消息延迟,毫秒级。
优点:即使磁盘损坏,消息丢失的非常少,且消息实时性不会受影响,因为 Master 宕机后,消费者仍然可以从 Slave 消费,此过程对应用透明。不需要人工干预。性能同多 Master 模式几乎一样。
缺点:Master 宕机,磁盘损坏情况,会丢失少量消息。
# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 &
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-a.properties >/dev/null 2>&1 &
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-b.properties >/dev/null 2>&1 &
#在机器 C,启动第一个 Slave
nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-a-s.properties >/dev/null 2>&1 &
# 在机器 D,启动第二个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-b-s.properties >/dev/null 2>&1 &
1.4.多Master多Slave模式,同步双写
每个 Master 配置一个 Slave,有多对Master-Slave,HA 采用同步双写方式,主备都写成功,向应用返回成功。
优点:数据与服务都无单点,Master宕机情况下,消息无延迟,服务可用性与数据可用性都非常高
缺点:性能比异步复制模式略低,大约低 10%左右,发送单个消息的 RT 会略高。目前主宕机后,备机不能自动切换为主机,后续会支持自动切换功能。
# 先启动 NameServer,例如机器 IP 为:”10.162.23.70:9876;10.162.23.71:9876”
# nohup ./bin/mqnamesrv >/dev/null 2>&1 &
# 在机器 A,启动第一个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-a.properties >/dev/null 2>&1 &
# 在机器 B,启动第二个 Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-b.properties >/dev/null 2>&1 &
# 在机器 C,启动第一个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-a-s.properties >/dev/null 2>&1 &
# 在机器 D,启动第二个 Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-b-s.properties >/dev/null 2>&1 &
注:
以上 Broker 与 Slave 配对是通过指定相同的brokerName 参数来配对,Master 的 BrokerId 必须是 0,Slave 的BrokerId 必须是大与 0 的数。另外一个 Master 下面可以挂载多个 Slave,同一 Master 下的多个 Slave 通过指定不同的 BrokerId 来区分。
除此之外,nameserver也需要集群。我们的环境采用多nameserver多master多slave异步复制模式
2.环境准备
角色 |
主机名 | IP | port | OS | 配置 |
Nameserver1 | INVOICE-HX-01 | 10.162.23.70 | 9876 | Redhat7.2 | 16C32G100G |
Nameserver2 | INVOICE-HX-02 | 10.162.23.71 | 9876 | Redhat7.2 | 16C32G100G |
Master1 | INVOICE-HX-01 | 10.162.23.70 | 10911 10922 | Redhat7.2 | 16C32G100G |
Master2 | INVOICE-HX-02 | 10.162.23.71 | 10911 10922 | Redhat7.2 | 16C32G100G |
Slave1 | INVOICE-HX-03 | 10.162.23.72 | 10911 10922 | Redhat7.2 | 16C32G100G |
Slave2 | INVOICE-HX-04 | 10.162.23.73 | 10911 10922 | Redhat7.2 | 16C32G100G |
moosefs安装前准备:
# RocketMQ软件包:alibaba-rocketmq-3.2.6.tar.gz
# JDK 1.7.79
# SELINUX :disable
# 安装路径:/opt/server/rocketmq
# 日志路径:/opt/var/logs/rocketmq
# 所有主机配置hosts
# cat /etc/hosts
10.162.23.70 INVOICE-HX-01
10.162.23.71 INVOICE-HX-02
10.162.23.72 INVOICE-HX-03
10.162.23.72 INVOICE-HX-04
注:hosts解析必须是真实的hostname
3.RocketMQ多M多异步复制集群安装
3.1.JDK安装
# 所有角色上安装
# 上传jdk-7u79-linux-x64.gz
# mkdir /opt/server -p
# tar zvxf jdk-7u79-linux-x64.gz -C /opt/server
# 修改 /etc/profile
# vi /etc/profile
# jdk
JAVA_HOME=/opt/server/jdk1.7.0_79
CLASSPATH=$JAVA_HOME/jre/lib/rt.jar:.
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH CLASSPATH
# source /etc/profile
# 检查java版本:
# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
3.2.RocketMQ 解压安装
# 所有角色上安装
# 上传alibaba-rocketmq-3.2.6.tar.gz
# tar zxvf alibaba-rocketmq-3.2.6.tar.gz
# mv alibaba-rocketmq /opt/server/rocketmq
#安装包bin下文件没有执行权限
# chmod +x /opt/server/rocketmq/bin/*
# 创建数据存储目录
# mkdir -p /opt/data/rocketmq/data/store/commitlog
# 创建日志目录
# mkdir -p /opt/var/logs/rocketmq/
# 修改日志目录
# cd /opt/server/rocketmq/conf
# sed -i 's#${user.home}/logs#/opt/var/logs/rocketmq#g' *.xml
# 查看修改
# grep logs *.xml
3.3.RocketMQ Nameserver启动
# nohup /opt/server/rocketmq/bin/mqnamesrv &
# 验证
# cat nohup.out
The Name Server boot success.
# netstat -anlp |grep 9876
tcp 0 0 :::9876 :::* LISTEN 23750/java
# 开机启动
# cat /etc/rc.local
# RocketMQ nameserver
nohup /opt/server/rocketmq/bin/mqnamesrv &
3.4.RocketMQ Master1 Slave1
# 修改master1(10.162.23.70)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=0
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 启动master1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 24220/java
tcp 0 0 :::10912 :::* LISTEN 24220/java
# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ master-a
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
# 修改slave1(10.162.23.72)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=1
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
# 启动salve1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 23774/java
tcp 0 0 :::10912 :::* LISTEN 23774/java
# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ slave-a
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
3.5.RocketMQ Master2 Slave2
# 修改master2(10.162.23.71)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=0
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 启动master2
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &
# 验证
# cat nohup.out
The Name Server boot success.
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-a, 10.162.23.71:10911] boot success.
# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 24220/java
tcp 0 0 :::10912 :::* LISTEN 24220/java
# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ master-b
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &
# 修改slave2(10.162.23.73)配置
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName=DefaultCluster
brokerName=broker-b
brokerId=1
namesrvAddr=10.162.23.70:9876;10.162.23.71:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
# 启动salve1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &
# 验证
# cat nohup.out
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-b, 10.162.23.73:10911] boot success. and name server is 10.162.23.70:9876;10.162.23.71:9876
# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 23774/java
tcp 0 0 :::10912 :::* LISTEN 23774/java
# 开机启动(nameserver必须为启动状态,建议手动启动)
# cat /etc/rc.local
# RocketMQ slave-b
nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &
注:服务停止直接采用kill java 进程的方法
3.6.RocketMQ nameserver 接口
10.162.23.70:9876;10.162.23.71:9876