mirrors
/
system-design-primer
mirror of https://github.com/donnemartin/system-design-primer.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add transactional messaging section with Outbox Pattern explanation

pull/1049/head
Ben Meehan 2025-01-13 14:16:06 +05:30
parent 40d5d2edcc
commit e016ca2d01
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
README.md
View File

@ -1356,6 +1356,24 @@ Tasks queues receive tasks and their related data, runs them, then delivers thei
If queues start to grow significantly, the queue size can become larger than memory, resulting in cache misses, disk reads, and even slower performance. [Back pressure](http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html) can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. Once the queue fills up, clients get a server busy or HTTP 503 status code to try again later. Clients can retry the request at a later time, perhaps with [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff).
### Transactional Messaging
Transactional messaging ensures reliable delivery and processing of messages while maintaining consistency across systems, especially in distributed architectures. It is critical for systems where operations must succeed or fail as a unit. [The Outbox Pattern](https://www.decodable.co/blog/revisiting-the-outbox-pattern) is a design to handle distributed transactions in an eventually consistent way. It ensures that messages are not lost even if a failure occurs while performing operations across multiple systems.
* The application writes a business change and the corresponding event (message) to an outbox table in the same database transaction.
* The transaction is committed, ensuring both the business operation and the event are saved together.
* A background worker periodically reads unprocessed messages from the outbox table and publishes them to the message queue.
* Upon successful delivery, the worker marks the messages as processed to avoid duplicates.
The database used for the outbox table must support ACID transactions to ensure atomicity, consistency, isolation, and durability. This ensures that both the business operation and the event write are treated as a single unit of work and are committed together reliably.
**Example:**
Consider an e-commerce system where a user places an order. As part of the transaction:
* The order's status is updated to "Placed" in the database.
* A corresponding "process payment" event is written to the outbox table within the same transaction.
Once the transaction is committed, a background worker reads the "process payment" event from the outbox table and publishes it to the payment service queue. This ensures that the payment service is notified about the new order only if the database update for the order status was successful. Similarly, the order status update happens only if the event is successfully written to the outbox table, maintaining consistency across the system.
### Disadvantage(s): asynchronism
* Use cases such as inexpensive calculations and realtime workflows might be better suited for synchronous operations, as introducing queues can add delays and complexity.