Write the business change and its event record in the same database transaction, then publish committed outbox records through a separate relay. This closes the dual-write gap between a database and a message broker: the business update cannot commit without the event being recorded, and a rolled-back update cannot be published. The relay can still publish an event more than once, so consumers must be idempotent.
What the outbox pattern guarantees—and what it does not
A service that updates a database and sends a message has two separate writes. If the database commits but message publication fails, downstream services never hear about the change. If the message is sent first and the database transaction rolls back, they may act on a change that never happened. The transactional outbox puts the business update and event record inside one local database transaction; a relay publishes only records that committed. AWS Prescriptive Guidance describes the pattern and example architectures.
This makes the database transaction the atomicity boundary. It does not create a distributed transaction with the broker or with other services, nor does it by itself guarantee exactly-once delivery. A relay can publish successfully and fail before recording that success, so it may publish the same event again on retry. Design for at-least-once publication and deduplicate downstream.
Design the outbox record
Keep enough information to identify, route, order, and recover an event. A relational outbox commonly includes these fields:
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
| Field | Purpose |
|---|---|
| Event ID | A globally unique, stable identifier retained across retries; consumers can use it to detect duplicates. |
| Aggregate or entity ID | Identifies the business object the event concerns and provides a key for per-aggregate ordering. |
| Event type | Names the kind of change so the relay or consumer can route and interpret it. |
| Payload | Contains the event data needed by downstream consumers. |
| Creation timestamp | Records when the application created the event. |
| Delivery and retry metadata | For a polling relay, tracks state and supports retries, monitoring, and cleanup. A CDC design may rely on the database change stream rather than updating a delivered flag. |
| Sequence or ordering metadata | Include when consumers need to process events for one aggregate in order; do not treat it as a promise of global ordering. |
Choose a payload contract deliberately: it should describe the event consumers need, rather than expose an accidental snapshot of internal database columns. Keep the event ID and aggregate ID stable, and define how event types and payloads evolve as the application changes.
Implement a relational outbox with polling
Polling is a practical starting point when a worker can periodically claim pending rows. The SQL below is schematic, not portable copy-and-paste syntax: transaction, locking, and JSON types vary by database.
BEGIN;
UPDATE business_entity
SET state = :new_state
WHERE id = :entity_id;
INSERT INTO outbox (
event_id, aggregate_id, event_type, payload, created_at,
delivery_status, retry_count
) VALUES (
:event_id, :entity_id, :event_type, :payload, CURRENT_TIMESTAMP,
'pending', 0
);
COMMIT;The application must execute both writes in the same transaction and treat failure of either write as failure of the operation. Generate the event ID before inserting the outbox row so it remains available as the identity for subsequent publication attempts.
- Create the outbox table. Add the fields above, define a uniqueness constraint for event IDs, and index the fields your worker uses to find pending records. Decide retention and replay requirements before defining cleanup.
- Write business state and event together. In the application transaction, perform the domain change and insert its corresponding event record. Commit once both are successful.
- Claim a batch of committed pending rows. Have workers coordinate so two workers do not concurrently process the same row. Use the database’s supported claim or locking strategy; exact SQL differs by engine. Apply batch limits and back-pressure so the relay does not overwhelm the database or broker.
- Publish each event. Send the stored event ID, type, aggregate key, and payload to the destination. If per-aggregate order matters, use the sequence or ordering metadata and a routing strategy that preserves order for that aggregate.
- Record success only after publication is acknowledged. Mark the row delivered after the broker acknowledges the send. If the worker crashes between acknowledgement and this update, it will retry and may publish a duplicate; that is expected and handled by idempotent consumers.
- Retry failures and surface exhausted work. Track attempts and last error, use bounded retry policy, and route persistent failures to a dead-letter or equivalent operational workflow. Alert on growing pending counts and old unprocessed records.
For concurrent workers, a claim must be safe under the database’s transaction and locking semantics. Avoid holding a database transaction open while waiting on a slow broker: claim or lease work, publish outside the long-running transaction where appropriate, then update delivery state. The exact lease and recovery design depends on the database and worker model.
Choose polling or change data capture
Both approaches publish only changes committed in the source database, but they shift operational work to different places. There is no universal latency or throughput winner independent of the database, workload, connector, and broker.
| Consideration | Polling relay | CDC relay |
|---|---|---|
| How it reads events | A worker queries and claims pending outbox rows. | A connector reads committed table changes from the database change stream or log. |
| Atomicity boundary | The business write and outbox insert share the database transaction. | The same transaction is the source of truth; CDC observes the committed outbox change. |
| Operational work | Worker coordination, locking or leasing, retries, cleanup, and back-pressure. | Connector and broker operations, database log retention, schema compatibility, and recovery monitoring. |
| Typical fit | A simpler architecture for moderate workloads when periodic dispatch is acceptable. | Often useful when lower-latency change propagation or high event volume makes polling overhead undesirable. |
| Delivery handling | Retries can republish after a crash or uncertain acknowledgement. | Connector restarts and downstream retries can also result in duplicate processing. |
Choose polling if operating a small worker is the more manageable option and the expected dispatch delay is acceptable. Choose CDC when the team can operate the connector and its log-retention and recovery requirements, and the workload benefits from streaming changes. In either case, preserve the event ID through publication and use consumer-side deduplication.
Rank #3
Adapt the pattern to DynamoDB, Kafka, and Debezium
DynamoDB Streams and EventBridge Pipes
AWS documents a DynamoDB design in which the order update and event information are stored atomically in DynamoDB, with DynamoDB Streams and Lambda used in a CDC-style flow. Its EventBridge Pipes example routes changes onward to consumers. This is a different storage implementation from a relational outbox table, but it retains the essential boundary: the data change and the information needed to emit the event must be written together. Consult the AWS implementation guidance for its RDS/SQS and DynamoDB examples before selecting a specific service configuration.
Kafka as the destination
Kafka can be the broker to which a polling relay or CDC pipeline publishes. Treat Kafka publication as the relay’s external side effect, not as part of the application’s database transaction. Carry the stable event ID and aggregate key into the record so consumers can deduplicate and, where needed, partition or process events consistently per aggregate. The outbox solves the database-to-publication gap; consumer idempotency and any ordering requirements remain application responsibilities.
Debezium Outbox Event Router
Debezium’s Outbox Event Router is designed to capture outbox-table changes and transform them for downstream consumers. This can remove the need for an application-managed polling loop, but it introduces connector, schema, log-retention, and broker operations. Define the outbox schema and event routing contract to match the router configuration, then monitor connector lag and recovery as part of the production design.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Make consumers idempotent and preserve required ordering
Every consumer that performs a consequential action should be safe when it receives the same event again. One approach is to store processed event IDs in the consumer’s own database and commit that record atomically with the consumer’s business effect. On receipt, the consumer checks the ID: if already processed, it skips the effect; otherwise it applies the effect and records the ID in one local transaction. A consumer-specific unique constraint on event ID can help enforce this under concurrent delivery.
Ordering is usually a per-aggregate concern, not a global guarantee. Include a sequence or equivalent ordering metadata if consumers must distinguish event order, and ensure the relay and destination routing preserve that order for the same aggregate. Do not infer order solely from timestamps, and do not assume unrelated aggregates need to block one another.
Test crash windows, operations, and cleanup
Test the boundaries where the process can fail rather than only the successful path. Verify that a rolled-back business transaction leaves no publishable event, while a committed transaction eventually reaches the relay. Exercise relay and consumer restarts to confirm that retries do not create duplicate business effects.
- Crash after the database commit but before the relay reads the event; confirm it remains discoverable.
- Crash after the broker acknowledges publication but before the relay records delivery; confirm a retry is harmless to consumers.
- Make publication fail repeatedly; confirm retry limits, dead-letter handling, alerts, and operator recovery are effective.
- Restart a CDC connector or disrupt access to the change log; verify recovery behavior and that log retention is sufficient for the expected interruption.
- Replay an event intentionally; verify consumers use the event identity to avoid repeating effects.
- Delete old outbox records only after they are no longer required for retry, audit, or replay. Set and document a retention policy that matches those needs.
Monitor pending-event age and volume, publish failures, retry counts, dead-letter volume, and CDC lag where applicable. Provide a replay procedure that preserves event identity, and establish who can use it and how consumer effects will remain safe.
When an outbox is not enough
The pattern coordinates one service’s database write with publication of an event. It does not make several independent services’ databases commit or roll back as one transaction. For a workflow that changes multiple services or data stores, use a saga or another explicit compensation strategy to define what happens when a later step fails.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
