concept
Delivery Guarantees (at-most-once / at-least-once / exactly-once)
Delivery Guarantees
Three guarantees you’ll see advertised by message brokers. One of them is, to put it mildly, misleading.
| Guarantee | What it does | When it’s right |
|---|---|---|
| At-most-once | Fire and forget. Send and don’t check. | Metrics where one dropped data point doesn’t matter. |
| At-least-once | Producer retries until ACK. If ACK is lost, message is resent. No loss, possible duplicates. | The default sane choice. Most systems use this. |
| Exactly-once | Advertised as the holy grail. Cannot be delivered across a network in general. See below. | Marketing slide deck. |
Why “exactly-once delivery” is a lie
The producer sends a message. The broker gets it. The broker sends back an ACK. The ACK vanishes on the wire. The producer now has to pick:
- Retry → maybe cause a duplicate.
- Give up → maybe lose the message.
The network doesn’t tell the producer which situation it’s in, ever. This is the two-generals-problem in distributed systems and it’s a result, not an implementation gap. Not Kafka, not RabbitMQ, not anything will magically fix it.
What vendors actually mean by “exactly-once”
When a vendor advertises exactly-once semantics, the runtime almost always combines:
- At-least-once delivery (you might get duplicates physically), plus
- Idempotent processing OR transactional writes at the consumer side.
The message might physically arrive twice, but the effect of processing it happens once because the consumer checks whether it has already seen that message and skips if so. Exactly-once delivery over a network: no. Exactly-once effect through idempotency|idempotent processing: yes.
Worth knowing specifically about Kafka: its “exactly-once” guarantee only covers what happens inside the Kafka cluster. The moment your consumer writes to an external database or calls an external API, you’re back to making that side idempotent yourself.
The practical rule
Assume at-least-once delivery. Build idempotent consumers. Every message handler checks if it has seen that event ID before and skips if it has. This single pattern prevents a category of bugs that, when they occur, tend to be nightmares to debug: double charges on payments, duplicate emails, inventory counts drifting in ways hard to reproduce.
If you do one thing after reading this: go check your message handlers. If they’re not idempotent, make them idempotent.
See also
- idempotency — the implementation pattern
- two-generals-problem — why exactly-once isn’t physically achievable
- dead-letter-queue — what to do with messages that keep failing despite retries
- pub-sub-vs-point-to-point — guarantees interact with delivery shape
- kafka, rabbitmq, sqs — what each broker advertises and what it actually means