一、问题
开发中遇到如下问题:在发送rocketmq消息的时候,包连接异常(RemotingConnectException)
连接broker地址为:10.100.2.0:10911 这一看就不像一个内网地址。
报错信息如下:
Caused by: org.apache.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [6005]ms, Topic: recordTopic, BrokersSent: [nb003, nb003, nb003]
See http://rocketmq.apache.org/docs/faq/ for further details.
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:665)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343)
at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:344)
at org.apache.rocketmq.spring.core.RocketMQTemplate.syncSend(RocketMQTemplate.java:555)
... 73 common frames omitted
Caused by: org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to 10.100.2.0:10911 failed
at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:394)
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:503)
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:487)
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:431)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:854)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:584)
... 76 common frames omitted
查看控制台:发现问题一致。
二、问题排查
先排查网络情况:ip addr
由上图发现,默认网卡的内网ip为172.xx.xx.xx (知道这个意思就行)
而实际报错里的 10.100.2.0 为之前安装K8S环境时候安装的网络插件 flannel 的ip段。故是broker 在启动时候网卡读错了。导致ip不对,从而导致连接异常。
三、解决方案
指定broker的ip地址,再启动broker
我的rocketmq的根路径为:/usr/local/rocketmq-all-4.9.1-bin-release
操作如下:
【172.xx.xx.xx改为你自己的ip哈】
[root@nb003 conf]# cd /usr/local/rocketmq-all-4.9.1-bin-release/conf
[root@nb003 conf]# echo "brokerIP1=172.xx.xx.xx" > broker.properties
[root@nb003 conf]# ls
2m-2s-async 2m-2s-sync 2m-noslave broker.conf broker.properties dledger logback_broker.xml logback_namesrv.xml logback_tools.xml plain_acl.yml tools.yml
# 先停止broker
[root@nb003 conf]# cd /usr/local/rocketmq-all-4.9.1-bin-release/bin
[root@nb003 bin]# ./mqshutdown broker
The mqbroker(2783) is running...
Send shutdown request to mqbroker(2783) OK
# 再启动broker,并指定配置文件(建议全路径)
[root@nb003 bin]# nohup sh mqbroker -n localhost:9876 -c /usr/local/rocketmq-all-4.9.1-bin-release/conf/broker.properties &
[1] 8705
[root@nb003 bin]# nohup: ignoring input and appending output to ‘nohup.out’
重新查看集群信息:ip已经正确。