Not a Drop More or Less Than What Was Sent

Opening Scene

A water delivery system that occasionally sends slightly more water than requested, due to a minor redundancy in its delivery mechanism, might be perfectly acceptable for most uses, but genuinely unacceptable for a precise, metered industrial process requiring exactly the specified amount, no more and no less. Delivery guarantees in a streaming system present this exact same genuine distinction in precision.

In Plain English

At-least-once delivery guarantees every event is delivered, but occasionally an event might be delivered more than once, typically due to a retry after an ambiguous failure. Exactly-once delivery guarantees every event is delivered precisely once, eliminating duplicates entirely, but requiring meaningfully more sophisticated coordination to achieve reliably. The right choice depends on whether a given consumer’s processing logic can safely tolerate occasional duplicates.

The Old Way

Before this distinction was widely and clearly understood as a deliberate design choice, delivery semantics were sometimes assumed rather than explicitly considered:

• Consumers were sometimes built without explicitly considering whether their processing logic could safely tolerate occasional duplicate events.
• There wasn’t yet a well-established practice of deliberately choosing between at-least-once and exactly-once semantics based on a genuine understanding of the tradeoff.
• Duplicate events sometimes caused subtle, difficult-to-diagnose bugs in systems that weren’t explicitly designed to handle them.

Assuming delivery semantics rather than deliberately choosing them is what explicit, informed delivery guarantee selection directly addresses.

What’s Changing (and Why AI Is the Reason)

1. Organizations increasingly choose deliberately between at-least-once and exactly-once semantics, based on genuine analysis of whether duplicate events could cause real problems for a specific consumer.
2. This connects directly to the consumer offset tracking covered in Article 9, since exactly-once semantics typically require careful coordination between offset commits and processing completion.
3. As AI training pipelines can sometimes be genuinely sensitive to duplicate data skewing a model’s learned patterns, exactly-once semantics have become an especially important consideration specifically for AI training data ingestion, even though the added coordination complexity is a genuine cost worth weighing.

The Metaphor, Fully Extended

The Water Utility	Cloud-Native Streaming Concept
Occasionally sending slightly more water than requested	At-least-once delivery occasionally delivering an event more than once
Perfectly acceptable for most everyday uses	Perfectly acceptable for consumers tolerant of occasional duplicates
Genuinely unacceptable for a precise, metered process	Genuinely unacceptable for consumers sensitive to duplicate processing
A genuine distinction in precision worth understanding	A genuine distinction in delivery guarantees worth understanding

Quick Recap

• At-least-once delivery guarantees every event arrives but may occasionally duplicate; exactly-once eliminates duplicates entirely.
• The right choice depends on whether a consumer’s processing logic can safely tolerate duplicates.
• Exactly-once semantics require meaningfully more sophisticated coordination to achieve reliably.
• AI training pipelines sensitive to duplicate data particularly benefit from exactly-once delivery guarantees.

Where This Fits in the Series

Article 12 covered the genuine distinction between delivery guarantee types. Article 13 turns to a related precision concern: the order water arrives at the tap.