From 492a2be89146fe294d6951e0f143394852e51d8d Mon Sep 17 00:00:00 2001 From: Donne Martin Date: Wed, 1 Mar 2017 20:42:13 -0800 Subject: [PATCH] Add Back pressure section --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index c1a5cc71..86d4db47 100644 --- a/README.md +++ b/README.md @@ -1428,3 +1428,7 @@ The user is not blocked and the job is processed in the background. During this Tasks queues receive tasks and their related data, runs them, then delivers their results. They can support scheduling and can be used to run computationally-intensive jobs in the background. **Celery** has support for scheduling and primarily has python support. + +### Back pressure + +If queues start to grow significantly, the queue size can become larger than memory, resulting in cache misses, disk reads, and even slower performance. [Back pressure](http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html) can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. Once the queue fills up, clients get a server busy or HTTP 503 status code to try again later. Clients can retry the request at a later time, perhaps with [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff).