# Throttling Pattern

The Throttling Pattern is a design pattern used to control the consumption of resources by an application instance, an individual tenant, or an entire service. It ensures that the system continues to function and meet service level agreements (SLAs), even when demand increases.

## 1. The Problem: Overload

If a system receives more requests than it can handle, it might crash or become unresponsive for all users.
*   **Scenario:** A sudden spike in traffic (e.g., Black Friday).
*   **Result:** CPU spikes to 100%, memory fills up, database connections run out. The system fails completely.

## 2. The Solution: Control the Flow

Throttling limits the number of requests a user or service can make within a specific timeframe. Unlike a Circuit Breaker (which stops traffic when a *fault* is detected), Throttling stops traffic when *capacity* is reached.

## 3. Throttling vs. Rate Limiting

These terms are often used interchangeably, but there is a subtle distinction in intent:

*   **Rate Limiting:** Often focuses on **security and fairness**. It prevents a single user (or bot) from abusing the API or hogging resources. (e.g., "You can only make 100 requests per hour").
*   **Throttling:** Often focuses on **system stability**. It protects the backend from being overwhelmed, regardless of who is sending the traffic. (e.g., "The database can only handle 1000 writes per second total").

## 4. Strategies

### Rejection (Hard Throttling)
The system immediately rejects the request with an error.
*   **HTTP Status:** `429 Too Many Requests`.
*   **Client Action:** The client should retry later (ideally using Exponential Backoff).

### Queueing (Soft Throttling)
The system accepts the request but puts it in a queue to be processed when resources become available.
*   **Pros:** No error for the user (just higher latency).
*   **Cons:** The queue itself consumes memory. If the queue fills up, you must eventually reject requests.

### Load Shedding / Degradation
The system disables non-essential features to save resources for critical ones.
*   **Example:** A video streaming service might switch everyone to 720p instead of 4K during peak hours to save bandwidth.

## 5. Implementation

Throttling can be implemented at multiple levels:

1.  **Application Level:** In code (using Token Bucket or Leaky Bucket algorithms).
2.  **Infrastructure Level:** Load Balancers (NGINX, HAProxy) or API Gateways (AWS API Gateway, Kong).
3.  **Database Level:** Many databases (like DynamoDB) have built-in read/write capacity units.

## 6. Backpressure

Backpressure is a mechanism where a downstream service (consumer) signals to an upstream service (producer) that it is overwhelmed and needs the producer to slow down.

*   **Push Model:** The producer sends data as fast as it can. If the consumer is slow, it crashes or runs out of memory buffering data.
*   **Pull Model (Backpressure):** The consumer requests data only when it is ready to process it. The producer waits until the consumer asks for more.

### Reactive Streams
Backpressure is a core concept in **Reactive Programming** (e.g., RxJava, Project Reactor, Akka). It allows systems to handle streams of data gracefully without buffering issues.

### TCP Flow Control
At the network layer, TCP uses a "sliding window" mechanism to implement backpressure. If the receiver's buffer is full, it tells the sender to stop sending packets.

[[programming/rate-limiting-algorithms]]
[[programming/circuit-breaker-pattern]]
[[programming/bulkhead-pattern]]
[[programming/system-design]]