James Phoenix
System design

Transactional outbox: two deliveries, one effect

The transactional outbox pattern stores a business change and its pending event in one database transaction. A relay publishes committed events later, while an idempotent consumer records each processed event with its business effect to make duplicate delivery safe.

Follow a real transaction boundary through rollback, a relay crash and duplicate publication, then see why the consumer must commit its dedupe key and business effect together.

By Film published 1:23 watch

Compare after the film
Loading film

One local transactionAn order and the event announcing it belong together. The transactional outbox writes the order and a pending event in one database transaction, while a separate relay delivers the event later.

Narrated with James Phoenix's AI voice.

2
Atomic boundaries
2
Broker deliveries
1
Committed effects
1
Duplicates suppressed

The relay crash causes duplicate publication. The consumer commits 1 effect with its processed-event key. The comparison changes delivery count, not the local atomic boundary.

A database update and a message publication have separate failure boundaries. The transactional outbox places the business change and a pending event inside one database transaction. A separate relay publishes committed events later. This makes the local decision atomic: a rollback preserves neither write, while a commit preserves both.

The film follows a deliberately paced teaching trace. Its first transaction rolls back. Its second commits one order and one outbox event. The relay publishes that event, then crashes before updating the row to say it was sent. The broker has a message, but the database still has an unsent event. After restart, the relay can publish it again.

The two broker deliveries have different delivery identifiers and the same event identifier. The consumer uses a durable processed-event ledger scoped to its own consumer identity. It commits the ledger entry and its business effect in the same local transaction. A consumer rollback leaves neither committed. A duplicate whose event key already exists skips the effect. The illustrated run ends with two deliveries and one effect.

These assertions were checked against local SQLite transactions, beyond the visual model. The verification uses separate producer, broker and consumer database files. A separate relay process exits immediately after its broker insert commits; a fresh relay process then republishes the still-unsent event. Producer rollback, consumer rollback after a key insert, atomic consumer commit and duplicate suppression are checked against the resulting rows. The broker file stands in for durable message storage; this is a process-exit test, not an operating-system crash or power-loss experiment.

After the film, compare a crash after publication with a relay that completes normally. The crash changes the number of deliveries, while the atomic consumer keeps one committed effect. The narrated example always includes the crash, so the explanation and its evidence stay aligned.

This is not an exactly-once transport claim. The consumer's guarantee requires the dedupe record and the effect to share a durable transaction boundary. An external email or charge cannot simply join a local database transaction; it needs a separately designed idempotency or delivery strategy. Event ordering, dedupe retention, parallel relay coordination, poison messages and recovery from permanent failure are additional concerns outside this trace.

The maths

One producer transaction
commit⁡(B,O)∈{(0,0),(1,1)}\operatorname{commit}(B,O)\in\{(0,0),(1,1)\}

B is the business write and O is its outbox event. The display counts committed writes, not transient statements inside the transaction.

Delivery identity and event identity differ
d1≠d2,event⁡(d1)=event⁡(d2)d_1\ne d_2,\qquad \operatorname{event}(d_1)=\operatorname{event}(d_2)

Republishing creates another delivery of the same logical event.

Atomic consumer dedupe
ΔE(c,e)={1(c,e)∉L0(c,e)∈L\Delta E(c,e)=\begin{cases}1&(c,e)\notin L\\0&(c,e)\in L\end{cases}

The effect increment and insertion into ledger L commit together. A rollback commits neither.

Transcript

The narration, chapter by chapter. A timestamp opens the film at that moment.

0:00One local transaction
An order and the event announcing it belong together. The transactional outbox writes the order and a pending event in one database transaction, while a separate relay delivers the event later.
0:11Both or neither
First, this transaction rolls back. Both staged writes disappear together: no saved order, no outbox event, and nothing for the relay to publish. A second transaction commits, so both records become durable at the same boundary.
0:25The publication gap
The relay reads the committed event and publishes it. Now it crashes before marking the outbox row sent. The broker already holds the message, but the database still says unsent; those actions were never one atomic transaction.
0:37A second atomic boundary
The consumer also needs an atomic boundary. Here a consumer transaction rolls back: neither its processed-event key nor its business effect survives. On retry, both commit together, so a later delivery can recognise completed work.
0:51Same event, new delivery
After the relay restarts, the unsent row is published again. This second delivery carries the same event identifier. The consumer ledger already contains that identifier, so the duplicate adds no new effect: two deliveries, one committed effect.
1:05What the guarantee needs
That is reliable processing through two local atomic boundaries, with duplicates allowed between them. The guarantee depends on a durable ledger committed with the effect, so an external charge or email needs its own idempotency strategy; transport alone is not exactly once.

Also available as captions (WebVTT), the video file and a Markdown copy of this page.

Sources and model assumptions

Follow the original mechanism behind this explainer. The interactive examples identify their toy data and simplifying assumptions above.

More visualisations

Embed this film

Paste this into a post, a course page or a newsletter. The film plays in place, with the same controls as here, and credits the source.

Building with language models?

These explainers come out of the work. If you want the same thinking applied to your own system, that is what I do.

See how I can help