mirrors
/
system-design-primer
mirror of https://github.com/donnemartin/system-design-primer.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add idempotency section to asynchronism

pull/558/head
Jeffrey Scott Keone Payne 2021-08-16 23:46:52 -07:00
parent 4e70c54a5b
commit c3267e1052
1 changed files with 7 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@ -161,6 +161,7 @@ Review the [Contributing Guidelines](CONTRIBUTING.md).
* [Message queues](#message-queues) * [Message queues](#message-queues)
* [Task queues](#task-queues) * [Task queues](#task-queues)
* [Back pressure](#back-pressure) * [Back pressure](#back-pressure)
* [Idempotent operations](#idempotent-operations)
* [Communication](#communication) * [Communication](#communication)
* [Transmission control protocol (TCP)](#transmission-control-protocol-tcp) * [Transmission control protocol (TCP)](#transmission-control-protocol-tcp)
* [User datagram protocol (UDP)](#user-datagram-protocol-udp) * [User datagram protocol (UDP)](#user-datagram-protocol-udp)
@ -1356,6 +1357,12 @@ Tasks queues receive tasks and their related data, runs them, then delivers thei
If queues start to grow significantly, the queue size can become larger than memory, resulting in cache misses, disk reads, and even slower performance. [Back pressure](http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html) can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. Once the queue fills up, clients get a server busy or HTTP 503 status code to try again later. Clients can retry the request at a later time, perhaps with [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff). If queues start to grow significantly, the queue size can become larger than memory, resulting in cache misses, disk reads, and even slower performance. [Back pressure](http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html) can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. Once the queue fills up, clients get a server busy or HTTP 503 status code to try again later. Clients can retry the request at a later time, perhaps with [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff).
### Idempotent operations
It is important to understand the benefits of [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning) operations, especially when using message or task queues that do not guarantee *exactly once* processing. Many queueing systems guarantee *at least once* message delivery or processing. These systems are not completely synchronized, for instance, across geographic regions, which simplifies some aspects of their implemntation or design. Designing the operations that a task queue executes to be idempotent allows one to use a queueing system that has accepted this design trade-off.
**[Google Pub/Sub](https://cloud.google.com/pubsub)** is a highly scalable, reliable, hosted messaging platform, but, on its own, does not guarantee *exactly once* delivery.
### Disadvantage(s): asynchronism ### Disadvantage(s): asynchronism
* Use cases such as inexpensive calculations and realtime workflows might be better suited for synchronous operations, as introducing queues can add delays and complexity. * Use cases such as inexpensive calculations and realtime workflows might be better suited for synchronous operations, as introducing queues can add delays and complexity.