Problems / Design a product, medium

Build a log aggregation pipeline

Design a log pipeline from an empty canvas: agents on every server ship batches, every line is indexed for search within seconds, and nothing is ever lost.

Unlock it once for 15 credits; everything inside it is free (runs, changes, Submit, results, hints). Sign in with Google to start; new accounts get free credits.

What it must do

Scale

Request types to route: ship (90%, write), search (10%, read). Budget: $6,500 a month.

Estimate first

The test cases

You start from an empty canvas. When you run the tests, each one simulates your design with its own traffic and failures:

TestWhat it doesPasses when
Normal dayThe normal day: the traffic in the brief, nothing broken.Success ≥ 99.9%; p99 latency ≤ 150 ms (ship); p99 latency ≤ 500 ms (search); No lost events
An incident: debug logging everywhereAt 5 s an incident starts and every service logs at debug level: traffic triples for 8 s.Success ≥ 99%; p99 latency ≤ 300 ms (ship)
The log stream goes downAt 5 s the log stream (the Kafka cluster) crashes and takes 10 s to come back, while every agent keeps shipping.Success ≥ 99%; No lost events
A server diesAt 4 s one server of the first app service on the request path dies for 10 s.Success ≥ 99%
Budget: $6,500/monthThe normal day's design must cost at most $6,500 a month.Cost ≤ $6,500/month

Must include

More design a product problems