RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装

文章目录

RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装


RocketMQ的安装(单节点)

接下来,我们从无到有 ,搭建一个RocketMQ的环境吧,单节点走起 。


安装虚拟机(optional)

假如你没有云主机也没搭建本地虚拟机,那么推荐使用VMware Workstation Player 来搭建自己的虚机,Virtual Box 也不错,这两款体积都比较小,适合个人开发。

VMware Workstation 15.5 Player的安装,请移步: VMware-使用 VMware Workstation 15.5 Player 替换 VMware Workstation Pro

安装的时候,把网络提前启用,省去了后面的配置
RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装


JDK 和 RocketMQ

RocketMQ 是个纯java、分布式、队列模型的开源消息中间件,所以JDK必不可少了。

RocketMQ 版本及JDK的对应关系

RocketMQ 版本及JDK的对应关系 : 戳这里
RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装


版本选择

  • JDK版-1.8.0_221
  • RocketMQ版本:4.3.2 release

RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装

下载Binary 二进制版本的即可,安装方便。


JDK1.8.0_221 安装

JDK(rpm)安装: 戳这里 【rpm安装后,默认安装路径为/usr/java】

JDK (tar.gz) 安装: 戳这里


RocketMQ- 安装 、配置、启动、关闭

RocketMQ- 安装

解压zip包到指定目录

约定下安装目录: /usr/local/rocketmq ,没有的目录自行创建 。

将下载好的 rocketmq-all-4.3.2-bin-release.zip 使用unzip命令,解压到 /usr/local/rocketmq目录下 如下所示

[root@artisan rocketmq]# pwd
/usr/local/rocketmq
[root@artisan rocketmq]# ll
total 0
drwxr-xr-x. 6 root root 96 Nov  3 19:10 rocketmq-all-4.3.2-bin-release

环境变量设置

修改 /etc/profile ,检查jdk与rocketmq的环境变量是否配置 (这里配置的是全局变量,对所有用户生效,可以不在这里配置,在应用的脚本里设置也是可以的,这里为了方便后续的操作,添加到全局变量中)

/etc/profile 末尾增加如下内容(根据自己下载的软件信息和安装目录配置)

# JDK
export JAVA_HOME=/usr/java/jdk1.8.0_221-amd64
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
#rocketmq
export ROCKETMQ_HOME=/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release
export PATH=$ROCKETMQ_HOME/bin:$PATH

第二步: 刷新环境变量

source /etc/profile

看下是否生效了

[root@artisan ~]# echo $JAVA_HOME
/usr/java/jdk1.8.0_221-amd6
[root@artisan ~]# echo $ROCKETMQ_HOME
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release
[root@artisan ~]# 

OK,生效,与配置相同。


RocketMQ-配置

修改配置文件 broker.conf

位于$ROCKETMQ_HOME/conf目录下 ,启动的时候可以通过-c参数加载该配置文件

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.

#集群名称,可自定义
brokerClusterName=DefaultCluster
brokerName=broker‐a
brokerId=0
#rocketmq‐name服务地址,多个地址用;分开,不配置默认为localhost:9876
namesrvAddr=192.168.18.130:9876
deleteWhen=04
fileReservedTime=48
#主从角色SYNC_MASTER,ASYNC_MASTER,SLAV
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
##允许自动创建主题topi
autoCreateTopicEnable=true
#broker监听端口
listenPort=10911
#消息存储根路径
storePathRootDir=/data/rocketmq/store
#日志路径
storePathCommitLog=/data/rocketmq/store/commitlog

配置说明:

  • 1.namesrvAddr : NameServer地址,可以配置多个,用逗号分隔;

  • 2.brokerClusterName 所属集群名称,如果节点较多可以配置多个

  • 3.brokerName broker名称,master和slave使用相同的名称,表明他们的主从关系

  • 4.brokerId 0表示Master,大于0表示不同的slave

  • 5.deleteWhen 表示几点做消息删除动作,默认是凌晨4点

  • 6.fileReservedTime 在磁盘上保留消息的时长,单位是小

  • 7.brokerRole 有三个值:【SYNC_MASTER,ASYNC_MASTER,SLAVE】;同步和异步表示Master和Slave之间同步数据的机制;

  • 8.flushDiskTyp 刷盘策略,取值为:【ASYNC_FLUSH,SYNC_FLUSH】表示同步刷盘和异步刷盘;SYNC_FLUSH消息写入磁盘后才返回成功状态,ASYNC_FLUSH不需要;

  • 9.listenPort 启动监听的端口号

  • 10.storePathRootDir 存储消息的根目录

  • 11.storePathCommitLog 默认路径${user.home} \store\${commitlog}\${fileName}


jvm 内存设置(optional)

RocketMQ 默认给的jvm配置是很大的,我这里使用了虚拟机,分配的内存只有1G,所以需要将内存设置的小一些,否则无法启动。

修改$ROCKETMQ_HOME/bin/runserver.sh$ROCKETMQ_HOME/bin/runbroker.sh中JAVA_OPT关于内存的设置

runserver.sh

[root@artisan bin]# cat -n runserver.sh 
     1	#!/bin/sh
     2	
     3	# Licensed to the Apache Software Foundation (ASF) under one or more
     4	# contributor license agreements.  See the NOTICE file distributed with
     5	# this work for additional information regarding copyright ownership.
     6	# The ASF licenses this file to You under the Apache License, Version 2.0
     7	# (the "License"); you may not use this file except in compliance with
     8	# the License.  You may obtain a copy of the License at
     9	#
    10	#     http://www.apache.org/licenses/LICENSE-2.0
    11	#
    12	# Unless required by applicable law or agreed to in writing, software
    13	# distributed under the License is distributed on an "AS IS" BASIS,
    14	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    15	# See the License for the specific language governing permissions and
    16	# limitations under the License.
    17	
    18	#===========================================================================================
    19	# Java Environment Setting
    20	#===========================================================================================
    21	error_exit ()
    22	{
    23	    echo "ERROR: $1 !!"
    24	    exit 1
    25	}
    26	
    27	[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
    28	[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
    29	[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"
    30	
    31	export JAVA_HOME
    32	export JAVA="$JAVA_HOME/bin/java"
    33	export BASE_DIR=$(dirname $0)/..
    34	export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
    35	
    36	#===========================================================================================
    37	# JVM Configuration  修改此处的内存大小,默认为4g,一般我们的虚拟机内存都设置的比较小 , 根据自己机器的配置合理设置
    38	#===========================================================================================
    39	JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=64m -XX:MaxMetaspaceSize=64m"
    40	JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8  -XX:-UseParNewGC"
    41	JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
    42	JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
    43	JAVA_OPT="${JAVA_OPT}  -XX:-UseLargePages"
    44	JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
    45	#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
    46	JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
    47	JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
    48	
    49	$JAVA ${JAVA_OPT} $@
[root@artisan bin]# 


runbroker.sh

[root@artisan bin]# cat -n runbroker.sh 
     1	#!/bin/sh
     2	
     3	# Licensed to the Apache Software Foundation (ASF) under one or more
     4	# contributor license agreements.  See the NOTICE file distributed with
     5	# this work for additional information regarding copyright ownership.
     6	# The ASF licenses this file to You under the Apache License, Version 2.0
     7	# (the "License"); you may not use this file except in compliance with
     8	# the License.  You may obtain a copy of the License at
     9	#
    10	#     http://www.apache.org/licenses/LICENSE-2.0
    11	#
    12	# Unless required by applicable law or agreed to in writing, software
    13	# distributed under the License is distributed on an "AS IS" BASIS,
    14	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    15	# See the License for the specific language governing permissions and
    16	# limitations under the License.
    17	
    18	#===========================================================================================
    19	# Java Environment Setting
    20	#===========================================================================================
    21	error_exit ()
    22	{
    23	    echo "ERROR: $1 !!"
    24	    exit 1
    25	}
    26	
    27	[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
    28	[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
    29	[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"
    30	
    31	export JAVA_HOME
    32	export JAVA="$JAVA_HOME/bin/java"
    33	export BASE_DIR=$(dirname $0)/..
    34	export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
    35	
    36	#===========================================================================================
    37	# JVM Configuration   默认需要内存大小为8g   根据自己机器的配置合理设置
    38	#===========================================================================================
    39	JAVA_OPT="${JAVA_OPT} -server -Xms256m  -Xmx256m -Xmn128m"
    40	JAVA_OPT="${JAVA_OPT} -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8"
    41	JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
    42	JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
    43	JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
    44	JAVA_OPT="${JAVA_OPT} -XX:+AlwaysPreTouch"
    45	JAVA_OPT="${JAVA_OPT} -XX:MaxDirectMemorySize=15g"
    46	JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages -XX:-UseBiasedLocking"
    47	JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
    48	#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
    49	JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
    50	JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
    51	
    52	numactl --interleave=all pwd > /dev/null 2>&1
    53	if [ $? -eq 0 ]
    54	then
    55		if [ -z "$RMQ_NUMA_NODE" ] ; then
    56			numactl --interleave=all $JAVA ${JAVA_OPT} $@
    57		else
    58			numactl --cpunodebind=$RMQ_NUMA_NODE --membind=$RMQ_NUMA_NODE $JAVA ${JAVA_OPT} $@
    59		fi
    60	else
    61		$JAVA ${JAVA_OPT} $@
    62	fi
[root@artisan bin]# 


修改配置文件 broker.conf,配置brokerIP

位于$ROCKETMQ_HOME/conf目录下

# 指定broker的ip为主网卡的地址
brokerIP1=192.168.18.130

RocketMQ-启动namesrv

先启动namesrv ,然后启动broker

$ROCKETMQ_HOME/bin下执行 nohup ./mqnamesrv -n 192.168.18.130:9876&

[root@artisan bin]# pwd
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release/bin
[root@artisan bin]# nohup ./mqnamesrv -n 192.168.18.130:9876&
[1] 15588
[root@artisan bin]# nohup: ignoring input and appending output to ‘nohup.out’
tail -f nohup.out 
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON

[root@artisan bin]# jps  查看启动的java进程
15591 NamesrvStartup
20968 Jps
[root@artisan bin]# 



RocketMQ-启动broker

先启动namesrv ,然后启动broker

不指定启动配置文件

$ROCKETMQ_HOME/bin下执行

[root@artisan bin]#nohup sh ./mqbroker ‐n 192.168.18.130:9876 autoCreateTopicEnable=true&

指定启动配置配置文件

-c 参数 加载配置文件

$ROCKETMQ_HOME/bin下执行 nohup ./mqbroker -n 192.168.18.130:9876 -c ../conf/broker.conf autoCreateTopicEnable=true &

  • 192.168.18.130:9876为nameserver,broker要注册到该namesrv 注册中心
  • autoCreateTopicEnable=true(允许创建topic),如果broker.conf配置了autoCreateTopicEnable=true,启动命令中可以省略
[root@artisan bin]# pwd
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release/bin
[root@artisan bin]# nohup ./mqbroker -n 192.168.18.130:9876 -c ../conf/broker.conf autoCreateTopicEnable=true &
[2] 21072
[root@artisan bin]# nohup: ignoring input and appending output to ‘nohup.out’
tail -f nohup.out 
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON
The broker[brokerâa, 192.168.18.130:10911] boot success. serializeType=JSON and name server is 192.168.18.130:9876


^C
[root@artisan bin]# jps   查看启动的java进程
21076 BrokerStartup
15591 NamesrvStartup
21160 Jps
[root@artisan bin]# 


查看broker启动配置

$ROCKETMQ_HOME/bin下执行 ./mqbroker -m

[root@artisan bin]# pwd
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release/bin

[root@artisan bin]# ./mqbroker -m
2019-11-03 21\:29\:28 INFO main - namesrvAddr=
2019-11-03 21\:29\:28 INFO main - brokerIP1=192.168.18.130
2019-11-03 21\:29\:28 INFO main - brokerName=artisan
2019-11-03 21\:29\:28 INFO main - brokerClusterName=DefaultCluster
2019-11-03 21\:29\:28 INFO main - brokerId=0
2019-11-03 21\:29\:28 INFO main - autoCreateTopicEnable=true
2019-11-03 21\:29\:28 INFO main - autoCreateSubscriptionGroup=true
2019-11-03 21\:29\:28 INFO main - rejectTransactionMessage=false
2019-11-03 21\:29\:28 INFO main - fetchNamesrvAddrByAddressServer=false
2019-11-03 21\:29\:28 INFO main - transactionTimeOut=6000
2019-11-03 21\:29\:28 INFO main - transactionCheckMax=15
2019-11-03 21\:29\:28 INFO main - transactionCheckInterval=60000
2019-11-03 21\:29\:28 INFO main - storePathRootDir=/root/store
2019-11-03 21\:29\:28 INFO main - storePathCommitLog=/root/store/commitlog
2019-11-03 21\:29\:28 INFO main - flushIntervalCommitLog=500
2019-11-03 21\:29\:28 INFO main - commitIntervalCommitLog=200
2019-11-03 21\:29\:28 INFO main - flushCommitLogTimed=false
2019-11-03 21\:29\:28 INFO main - deleteWhen=04
2019-11-03 21\:29\:28 INFO main - fileReservedTime=72
2019-11-03 21\:29\:28 INFO main - maxTransferBytesOnMessageInMemory=262144
2019-11-03 21\:29\:28 INFO main - maxTransferCountOnMessageInMemory=32
2019-11-03 21\:29\:28 INFO main - maxTransferBytesOnMessageInDisk=65536
2019-11-03 21\:29\:28 INFO main - maxTransferCountOnMessageInDisk=8
2019-11-03 21\:29\:28 INFO main - accessMessageInMemoryMaxRatio=40
2019-11-03 21\:29\:28 INFO main - messageIndexEnable=true
2019-11-03 21\:29\:28 INFO main - messageIndexSafe=false
2019-11-03 21\:29\:28 INFO main - haMasterAddress=
2019-11-03 21\:29\:28 INFO main - brokerRole=ASYNC_MASTER
2019-11-03 21\:29\:28 INFO main - flushDiskType=ASYNC_FLUSH
2019-11-03 21\:29\:28 INFO main - cleanFileForciblyEnable=true
2019-11-03 21\:29\:28 INFO main - transientStorePoolEnable=false
[root@artisan bin]# 


关闭命令

先关闭 broker ,然后关闭 namesrv


正常退出 (推荐)

$ROCKETMQ_HOME/bin 调用mqshutdown 脚本执行

  • ./mqshutdown broker 关闭broker
  • ./mqshutdown namesrv 关闭namesrv
[root@artisan bin]# pwd
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release/bin
[root@artisan bin]# ./mqshutdown broker
The mqbroker(21076) is running...
Send shutdown request to mqbroker(21076) OK
[root@artisan bin]# ./mqshutdown namesrv
The mqnamesrv(15591) is running...
Send shutdown request to mqnamesrv(15591) OK
[root@artisan bin]# 
[1]-  Exit 143                nohup ./mqnamesrv -n 192.168.18.130:9876
[2]+  Exit 143                nohup ./mqbroker -n 192.168.18.130:9876 -c ../conf/broker.conf autoCreateTopicEnable=true
[root@artisan bin]# 
[root@artisan bin]# jps
22471 Jps
[root@artisan bin]# 

kill进程

ps ‐ef|grep rocketmq 查看pid(进程号)
kill ‐9 pid(进程号)

关闭防火墙

自己搭建的测试环境,防火墙关闭即可。生产环境考虑通过开放端口访问限制

如果客户端访出现 RemotingTooMuchRequestException: sendDefaultImpl call timeout

在客户端运行Producer时,可能会出现如上异常,这是因为从 Windows 上开发连接
虚拟机中的 nameServer 时要经过 Linux 系统的防火墙,而防火墙一般都会有超时的机制,在网络连接长时间不传输数据时,会关闭这个 TCP 的会话,关闭后再读写,就有可能导致这个异常。

centos7命令如下:

[root@artisan ~]# 关闭防火
[root@artisan ~]# systemctl stop firewalld.service

   
[root@artisan ~]# 禁止开机启
[root@artisan ~]# systemctl disable firewalld.service 


[root@artisan ~]# 查看状
[root@artisan ~]# systemctl status firewalld.service 查看状态
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

Nov 03 18:21:39 artisan systemd[1]: Starting firewalld - dynamic firewall daemon...
Nov 03 18:21:42 artisan systemd[1]: Started firewalld - dynamic firewall daemon.
Nov 03 18:21:42 artisan firewalld[896]: 2019-11-03 18:21:42 ERROR: Failed to create temporary file: [Errno 13] Permission denied: '/run/firewalld'
Nov 03 19:37:39 artisan systemd[1]: Stopping firewalld - dynamic firewall daemon...
Nov 03 19:37:39 artisan systemd[1]: Stopped firewalld - dynamic firewall daemon.
[root@artisan ~]# 查看状态
[root@artisan ~]# firewall-cmd --state
not running
[root@artisan ~]# 





关于autoCreateTopicEnable参数

autoCreateTopicEnable=true,是不是需要在启动的时候开启这个功能,一般情况都是设计阶段定好系统有几个queue, 系统上线前,由运维来创建。 我们这里测试用,为了防止报错,故设置为true ,自动创建Topic


遇到的问题

1. Vmware Player 虚拟机启动 PXE-E53:No boot filename received \ Operating System not found

RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装

ANSWER:

检查虚拟机的设置, 勾选启动时连接

刚用VMware Workstation 15.5 Player 替换 了VMware Workstation Pro , 结果Player 没勾选这个选项,所以从磁盘加载ISO的时候找不到。。。

RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装


上一篇:【三】vlc-android: native层播放流程分析


下一篇:Apache Kafka-初体验Kafka(03)-Centos7下搭建kafka集群