org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:775)

    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:726)

工作中遇到的kafka异常和解决办法 记录一下

原文地址https://blog.csdn.net/shibuwodai_/article/details/80678717

异常

异常的主要信息：

a) CommitFailedException

b) Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

其实如果我们对其中的参数，或是对消费的机制比较了解，这个问题就很好解决。当我看到这个异常，我很开心，因为我知道我能通过此异常了解一下Kafka Consumer 消费消息的大致过程。心态是好的~~~

其实现在看这个异常是说：该Consumer不能提交offset了，因为它已经出局了，是因为你的处理小时时间长于你要报告给server的时间。同时还告诉我们怎么处理：要么增加超时时间，要么减少每次poll回来的消息个数。

主要问题在于，何为session timeout？maximum size of batches？poll(timeout)中timeout什么意思？

处理过程

a) 找官网doc

版本：1.1.0

有效信息：

换成通俗易懂的人话：

下边这个例子如果理解不上，请通读全文后，再回来理解一下笔者的意思

------------------------------------分割线------------------------------------

------------------------------------分割线------------------------------------

通过上边的例子，我们大致清楚了max.poll.interval.ms？maximum size of batches？

max.poll.interval.ms：消费者最大心跳时间间隔

maximum size of batches：消费者每次获取消息的个数

什么时候发送心跳呢？是poll()方法被调用发送心跳吗？那poll(timeout)中timeout是什么意思呢？

官网对poll(timeout)中timeout的解释如下：

1.  
   Parameters:
2.  
   timeout - The time, in milliseconds, spent waiting in poll if data is not available in the buffer. If 0,

   returns immediately with any records that are available currently in the buffer,

   else returns empty. Must not be negative.

这个我费了很大力气都没有给它翻译成人话……