首页  

kafka消息投递语义     所属分类 kafka 浏览量 250
At most once 
Messages may be lost but are never redelivered
消息可能丢失但不会重复投递

At least once 
Messages are never lost but may be redelivered
消息不会丢失但可能重复投递
 
Exactly once 
this is what people actually want, each message is delivered once and only once
消息只投递一次

从0.11.0.0版本以后,Kafka生产者支持幂等投递选项,以保证即使消息被重新发送,日志中也不会有重复的条目。
为了实现这一点,broker给没用个生成者指定一个ID并且每条消息指定一个序列号。

站在消费者的角度,先保存位置后处理消息就是“最多一次”;先处理消息后保存位置就是“最少一次”;
至于“精确一次”,可以使用事务生产者来实现,即在同一个事务中接收并处理消息,将位置(offset)保存到另一个topic中。


The consumer's position is stored as a message in a topic, 
so we can write the offset to Kafka in the same transaction as the output topics receiving the processed data. 
If the transaction is aborted, the consumer's position will revert to its old value 
and the produced data on the output topics will not be visible to other consumers, depending on their "isolation level." 
In the default "read_uncommitted" isolation level, all messages are visible to consumers even if they were part of an aborted transaction, 
but in "read_committed," the consumer will only return messages from transactions which were committed 
(and any messages which were not part of a transaction).

上一篇     下一篇
CountDownLatch 与 CyclicBarrier

让自己更优秀的16条法则

kafka高可用机制简介

redis使用注意点

集群session处理

分布式数据库主键生成方案