利用9台机器搭建一个3分片的broker集群,一共有3个broker实例(3m),每个master有一个slave,使用同步刷盘模式。
1.引言
本文主要介绍如何在三台机器上搭建一个3m-3s的rocketmq的集群。2.环境准备
机器名 |
机器 |
机器配置 |
备注 |
nameserver1 |
192.168.11.1 |
4C8G |
|
nameserver2 |
192.168.11.2 |
4C8G |
|
nameserver3 |
192.168.11.3 |
4C8G |
|
broker11 |
192.168.11.4 |
8C16G | broker-a(master) |
broker12 |
192.168.11.5 |
8C16G | broker-a(slave) |
broker21 |
192.168.11.6 | 8C16G | broker-b(master) |
broker22 |
192.168.11.7 | 8C16G | broker-b(slave) |
broker31 |
192.168.11.8 |
8C16G |
broker-c(master) |
broker32 |
192.168.11.9 |
8C16G | broker-c(slave) |
2.1 配置hostname
这里设置是为了方便管理机器节点,也可以跳过此处。
#nameserver1上执行: hostnamectl set-hostname nameserver1 #nameserver2上执行: hostnamectl set-hostname nameserver2 #nameserver3上执行: hostnamectl set-hostname nameserver3 #broker11上执行: hostnamectl set-hostname broker11 #broker12上执行: hostnamectl set-hostname broker12 #broker21上执行: hostnamectl set-hostname broker21 #broker22上执行: hostnamectl set-hostname broker22 #broker31上执行: hostnamectl set-hostname broker31 #broker32上执行: hostnamectl set-hostname broker32
2.2 设置hosts
分别在所有broker机器和nameserver机器上执行:
cat >>/etc/hosts<<EOF 192.168.11.1 nameserver1 192.168.11.2 nameserver2 192.168.11.3 nameserver3 EOF
3. 具体步骤
3.1 下载rocketmq的编译好的二进制包(4.7.1版本)
wget https://mirror.bit.edu.cn/apache/rocketmq/4.7.1/rocketmq-all-4.7.1-bin-release.zip
3.2 解压缩到到/usr/local/mq目录,并创建软链接
#解压到/usr/local
unzip rocketmq-all-4.7.1-bin-release.zip -d /usr/local/ cd /usr/local/ #创立软链接 ln -s rocketmq-all-4.7.1-bin-release rocketmq
3.3 安装java
#Ubuntu: sudo apt install -y openjdk-8-jdk* #Centos: sudo yum install -y java-1.8.0-openjdk*
4.初始化环境
4.1 设置环境变量
#1. 设置JAVA_HOME
#如果是Ubuntu echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/'>> ~/.bashrc #如果是Centos echo 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/'>> ~/.bashrc
#2. 设置CLASSPATH,ROCKETMQ_HOME #无论是Ubuntu还是Centos都执行如下: #设置CLASSPATH echo 'export CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib'>> ~/.bashrc #设置ROCKETMQ_HOME echo 'export ROCKETMQ_HOME=/usr/local/rocketmq'>> ~/.bashrc
#添加到PATH echo 'export PATH=$PATH:$ROCKETMQ_HOME/bin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin'>> ~/.bashrc
4.2 加载环境变量
source ~/.bashrc
4.3 创建目录
#/data目录的空间至少为1T+ mkdir -pv {/data/logs,/data/rocketmq/store/commitlog}
4.4 修改rocketmq的主目录
sed -i 's#${user.home}#/data#g' $ROCKETMQ_HOME
/conf/*.xml
4.5 修改jvm参数
根据实际情况修改runserver.sh和runbroker.sh中的堆内存,如果机器够好,建议调大一点
4.6 修改Linux系统参数提高性能
cd $ROCKETMQ_HOME/bin sudo ./os.sh
5 启动
5.1 启动nameserver
1) 生成配置
分别在nameserver1,nameserver2,nameserver3上执行:
#进入Rockmq的bin目录 cd $ROCKETMQ_HOME/bin #生产nameserver的配置 cat >> namesrv.properties << EOF rocketmqHome=/usr/local/rocketmq kvConfigPath=/root/namesrv/kvConfig.json productEnvName=center clusterTest=false orderMessageEnable=false listenPort=9876 serverWorkerThreads=8 serverCallbackExecutorThreads=0 serverSelectorThreads=3 serverOnewaySemaphoreValue=256 serverAsyncSemaphoreValue=64 serverChannelMaxIdleTimeSeconds=120 serverSocketSndBufSize=65535 serverSocketRcvBufSize=65535 serverPooledByteBufAllocatorEnable=true useEpollNativeSelector=true EOF
2) 启动nameserver
cd $ROCKETMQ_HOME/bin nohup ./mqnamesrv -c namesrv.properties &
5.2 启动broker
分配配置六台broker
5.2.1 配置Rocket集群
broker11配置环境变量
cat >> ~/.bashrc << EOF Role=broker11 brokerId=0 GroupName=broker-a EOF
然后执行:source ~/.bashrc
broker11的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#master这里brokerRole设置为SYNC_MASTER brokerRole=SYNC_MASTER flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker11
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
broker12配置环境变量
cat >> ~/.bashrc << EOF Role=broker12 brokerId=1 GroupName=broker-a EOF
然后执行:source ~/.bashrc
broker12的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#slave就这里有区别,brokerRole设置为SLAVE brokerRole=SLAVE flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker12
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
broker21配置环境变量
cat >> ~/.bashrc << EOF Role=broker21 brokerId=0 GroupName=broker-b EOF
然后执行:source ~/.bashrc
broker21的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#master这里brokerRole设置为SYNC_MASTER brokerRole=SYNC_MASTER flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker21
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
broker22配置环境变量
cat >> ~/.bashrc << EOF Role=broker22 brokerId=1 GroupName=broker-b EOF
然后执行:source ~/.bashrc
broker22的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#slave就这里有区别,brokerRole设置为SLAVE brokerRole=SLAVE flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker22
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
broker31配置环境变量
cat >> ~/.bashrc << EOF Role=broker31 brokerId=0 GroupName=broker-c EOF
然后执行:source ~/.bashrc
broker31的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#master这里brokerRole设置为SYNC_MASTER brokerRole=SYNC_MASTER flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker31
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
broker32配置环境变量
cat >> ~/.bashrc << EOF Role=broker32 brokerId=1 GroupName=broker-c EOF
然后执行:source ~/.bashrc
broker32的配置生成
cd $ROCKETMQ_HOME cat >conf/dledger/qmq-${Role}.conf<<EOF brokerClusterName=QMQ brokerId=${brokerId}
#slave就这里有区别,brokerRole设置为SLAVE brokerRole=SLAVE flushDiskType=SYNC_FLUSH brokerName=${GroupName} namesrvAddr=nameserver1:9876;nameserver2:9876;nameserver3:9876 storePathRootDir=/data/rocketmq/${Role} storePathCommitLog=/data/rocketmq/${Role}/commitlog enableDLegerCommitLog=false #凌晨4点触发一次删除 deleteWhen = 04 diskMaxUsedSpaceRatio=50 #只保留最近1天的消息 fileReservedTime = 24 sendMessageThreadPoolNums=8 defaultTopicQueueNums=16 maxTransferCountOnMessageInMemory=1000 waitTimeMillsInSendQueue=1000 transientStorePoolEnable=false warmMapedFileEnable=false transferMsgByHeap=true maxTransferCountOnMessageInMemory=2000 maxTransferBytesOnMessageInMemory=2000*1024 #增加快速失败的时长 maxWaitTimeMillsInQueue=1000
#这里允许从slave进行消费
slaveReadEnable=true EOF
启动broker32
cd $ROCKETMQ_HOME/bin nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/qmq-${Role}.conf &
5.3 其它命令
Broker其它命令
1. 关闭broker和nameserver
sh mqshutdown broker sh mqshutdown namesrv
2. 查看broker是否启动
#查看进程 jps -m #查看端口 netstat -tupln|grep -E '309|409'
5.4 查看集群信息
通过控制台查询
博主:测试生财
座右铭:专注测试与自动化,致力提高研发效能;通过测试精进完成原始积累,通过读书理财奔向财务*。
csdn:https://blog.csdn.net/ccgshigao
博客园:https://www.cnblogs.com/qa-freeroad/
51cto:https://blog.51cto.com/14900374