（二）Kafka动态增加Topic的副本(Replication)

1. 查看topic的原来的副本分布

[hadoop@sdf-nimbus-perf ~]$ le-kafka-topics.sh --describe --topic http_zhixin_line1

Topic:http_zhixin_line1	PartitionCount:3	ReplicationFactor:1	Configs:

	Topic: http_zhixin_line1	Partition: 0	Leader: 7	Replicas: 7	Isr: 7

	Topic: http_zhixin_line1	Partition: 1	Leader: 8	Replicas: 8	Isr: 8

	Topic: http_zhixin_line1	Partition: 2	Leader: 9	Replicas: 9	Isr: 9

2. 增加Topic的副本的json文件的编写

vi addReplicas.json

{

    "version": 1,

    "partitions": [

        {

            "topic": "http_zhixin_line1",

            "partition": 0,

            "replicas": [

                7,

                1,

                2

            ]

        },

        {

            "topic": "http_zhixin_line1",

            "partition": 1,

            "replicas": [

                8,

                2,

                3

            ]

        },

        {

            "topic": "http_zhixin_line1",

            "partition": 2,

            "replicas": [

                9,

                3,

                4

            ]

        }

    ]

}

3. 执行topic增加副本操作

kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --execute

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --execute

Current partition replica assignment

{"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":2,"replicas":[9]},{"topic":"http_zhixin_line1","partition":1,"replicas":[8]},{"topic":"http_zhixin_line1","partition":0,"replicas":[7]}]}

Save this to use as the --reassignment-json-file option during rollback

Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":0,"replicas":[7,1,2]},{"topic":"http_zhixin_line1","partition":1,"replicas":[8,2,3]},{"topic":"http_zhixin_line1","partition":2,"replicas":[9,3,4]}]}

4. 查看执行的状态

kafka-reassign-partitions.sh --zookeeper  $ZK_CONNECT --reassignment-json-file addReplicas.json --verify

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --verify

Status of partition reassignment:

Reassignment of partition [http_zhixin_line1,0] completed successfully

Reassignment of partition [http_zhixin_line1,1] completed successfully

Reassignment of partition [http_zhixin_line1,2] completed successfully

5. 观察日志目录的数据同步情况，生产者，消费者的影响。

观察partitions-0的数据同步情况，由于partitions-0 增加的两个副本为1,2；

登陆 broker 1 和 broker 2所在的服务器；

cd 副本数据所在的目录，观察日志增长情况：

ls /data/hadoop/data*/kafka/log

borker1 和 broker2 所在日志目录都有日志文件生成。borker1 和 borker2的副本数据是同时同步完成的。

观察生产者：没有影响。

观察消费者：没有影响。