At a glance
- Did you know
- An in-memory cache like Redis usually answers in well under a millisecond inside one region.
- Rule of thumb
- A cold cache sends every read to the database: warm it before traffic arrives.
- Watch out
- A flush or restart causes a miss storm on the database behind it.
Problems that start with it
- Read-heavy product catalog 2,000 req/s, 90% reads through a cache. At 20 s the cache is flushed.
- URL shortener 4,000 req/s, 95% redirects. Popular links are far more popular than the rest.
- YouTube end to end 20,000 requests/s routed by path: watch, search, comments and uploads. Metadata cache in front of Vitess MySQL, recommendations, view events into Kafka, uploads to storage and a transcoding queue.
- Black Friday spike A shop handles 1,500 req/s comfortably. At 10 s, Black Friday traffic triples for 15 seconds.
- CPU-bound API The API renders pages on the server: 6 ms of CPU per request on 2 vCPUs per replica, at 1,500 req/s.
- The slow ramp Traffic grows from 800 to 2,400 req/s over the run. The API is sized for today.
- Bursty traffic Average traffic is 700 req/s, but every 8 seconds a 2-second burst brings 4× that. The API queue is short.
- Autoscaling hits its ceiling The API runs on a container platform that autoscales from 2 to 3 instances. Traffic needs more.
And 41 more on the problems page.
Full lesson coming soon
Lessons are being added one component at a time: how it works, its settings, a live exercise to break and fix it, interview questions and a short check. See the Circuit breaker lesson for what one looks like.