Kafka动态增加Topic的副本

一 需求描述

一个分区可以有多个副本,这些副本保存在不同的broker上,可实现kafka高可用。每个分区的副本中都会有一个作为Leader。当一个broker失败时,Leader在这台broker上的分区都会变得不可用,kafka会自动移除Leader,再其他副本中选一个作为新的Leader。

假如我们目前kafka集群环境中的topic没有副本,可通过如下方法增加topic的副本

二 操作步骤

2.1 查看某个topic当前的分区及副本信息

[root@Centos7-Mode-V8 kafka]# bin/kafka-topics.sh --describe --zookeeper 192.168.144.247:3292,192.168.144.251:3292,192.168.144.253:3292 --topic mubai

Topic:mubai PartitionCount:3 ReplicationFactor:1 Configs:

Topic: mubai Partition: 0 Leader: 0 Replicas: 0 Isr: 0

Topic: mubai Partition: 1 Leader: 1 Replicas: 1 Isr: 1

Topic: mubai Partition: 2 Leader: 0 Replicas: 0 Isr: 0

2.2 制定分区及副本分配策略

注意修改下面文件中的topic为自己的topic,注意修改里面的partition,replicas,值需要与上面2.1步骤中查询出来的自己topic的现有值保持一致

[root@Centos7-Mode-V8 opt]# vi /opt/test.json

{

    "version": 1,

    "partitions": [

        {

            "topic": "mubai",

            "partition": 0,

            "replicas": [

                0,

                1,

                2

            ]

        },

        {

            "topic": "mubai",

            "partition": 1,

            "replicas": [

                1,

                2,

                0

            ]

        },

        {

            "topic": "mubai",

            "partition": 2,

            "replicas": [

                2,

                0,

                1

            ]

        }

    ]

}

2.3 扩副本

 [root@Centos7-Mode-V8 kafka]# bin/kafka-reassign-partitions.sh --zookeeper 192.168.144.247:3292,192.168.144.251:3292,192.168.144.253:3292 --reassignment-json-file /opt/test.json --execute

Current partition replica assignment

{"version":1,"partitions":[{"topic":"mubai","partition":0,"replicas":[0],"log_dirs":["any"]},{"topic":"mubai","partition":2,"replicas":[0],"log_dirs":["any"]},{"topic":"mubai","partition":1,"replicas":[1],"log_dirs":["any"]}]}

Save this to use as the --reassignment-json-file option during rollback

Successfully started reassignment of partitions.

2.4 验证副本是否添加成功

bin/kafka-topics.sh --describe --zookeeper 192.168.144.247:3292,192.168.144.251:3292,192.168.144.253:3292 --topic mubai

可以看到输出结果和我们上面规划的是一样的:

Kafka动态增加Topic的副本

 Kafka动态增加Topic的副本

 ————————————————

版权声明:本文为CSDN博主「shykevin」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。

原文链接:https://blog.csdn.net/shykevin/article/details/90103360

上一篇:2021年Java网络编程总结篇,美团阿里Java程序员晒工资被围观


下一篇:Hibernate插入、查询、删除操作 HQL