当一个group中,有consumer加入或者离开时,会触发partitions均衡.均衡的最终目的,是提升topic的并发消费能力.
1) 假如topic1,具有如下partitions: P0,P1,P2,P3
2) 加入group中,有如下consumer: C0,C1
3) 首先根据partition索引号对partitions排序: P0,P1,P2,P3
4) 根据(consumer.id + '-'+ thread序号)排序: C0,C1
5) 计算倍数: M = [P0,P1,P2,P3].size / [C0,C1].size,本例值M=2(向上取整)
6) 然后依次分配partitions: C0 = [P0,P1],C1=[P2,P3],即Ci = [P(i * M),P((i + 1) * M -1)]

kafka1.1.0  相关的日志记录

2019-12-30 18:28:46 [main] INFO  org.apache.kafka.clients.Metadata -Cluster ID: 1rp4v7GxSKmF2HazVpDAbg
2019-12-30 18:28:46 [main] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Discovered group coordinator localhost:9092 (id: 2147483647 rack: null)
2019-12-30 18:28:46 [main] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Revoking previously assigned partitions []
2019-12-30 18:28:46 [main] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] (Re-)joining group
2019-12-30 18:28:48 [main] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Successfully joined group with generation 14
2019-12-30 18:28:48 [main] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Setting newly assigned partitions [kafka_demo_test_topic-5, kafka_demo_test_topic-4]
2019-12-30 18:31:24 [main] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Revoking previously assigned partitions [kafka_demo_test_topic-5, kafka_demo_test_topic-4]
2019-12-30 18:31:24 [main] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] (Re-)joining group

2019-12-30 18:31:32 [main] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Successfully joined group with generation 15
2019-12-30 18:31:32 [main] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -[Consumer clientId=consumer-1, groupId=kafkademo_test_group] Setting newly assigned partitions [kafka_demo_test_topic-2, kafka_demo_test_topic-1, kafka_demo_test_topic-0, kafka_demo_test_topic-5, kafka_demo_test_topic-4, kafka_demo_test_topic-3]