At a glance
- Did you know
- Little's law: throughput ≈ concurrency ÷ latency. 200 slots at 50 ms serve about 4,000 req/s.
- Rule of thumb
- Keep retries under about 10% of calls, or a slow dependency turns into a retry storm.
- Watch out
- Slots and queue fill up when a dependency slows; CPU at 100% crashes replicas.
Problems that start with it
- Read-heavy product catalog 2,000 req/s, 90% reads through a cache. At 20 s the cache is flushed.
- Slow database rescue The database answers reads in 9 seconds. The API gives up after 5 seconds and retries twice.
- Payment webhook processor 500 webhooks/s through an at-least-once queue. 0.2% of acknowledgements are lost.
- URL shortener 4,000 req/s, 95% redirects. Popular links are far more popular than the rest.
- Event pipeline catching up 2,000 events/s into a 6-partition stream with 30 s retention. 3 consumers.
- RAG support assistant 40 questions/s, 4,000-token prompts, a 6M tokens/min quota and a 25% answer-cache hit rate.
- Edge defence: survive a DDoS 1,500 req/s of real users. At 15 s a 20,000 req/s flood hits for 15 s. Nothing stands in front of the load balancer.
- YouTube end to end 20,000 requests/s routed by path: watch, search, comments and uploads. Metadata cache in front of Vitess MySQL, recommendations, view events into Kafka, uploads to storage and a transcoding queue.
And 264 more on the problems page.
Full lesson coming soon
Lessons are being added one component at a time: how it works, its settings, a live exercise to break and fix it, interview questions and a short check. See the Circuit breaker lesson for what one looks like.