Problems / Real world, medium

Deep research: from launch build to real traffic

A multi-agent research product on its first servers: the lead agent fans each question out to sub-agents that search and browse the web, and the browser tool, sub-agent pool and model capacity are an eighth of their real size. Questions arrive one a second and each takes minutes. One in ten fails.

Solve it in the labTake it as a mock interview

Unlock it once for 15 credits; everything inside it is free (runs, changes, Submit, results, hints). Sign in with Google to start; new accounts get free credits.

Your goals

Grow it into the real architecture: every goal must hold at 1 requests a second. You may add components, connect them and resize anything.

Test cases

Like a coding problem, a solution has to pass every case, not just the one it was built for. Submit in the lab runs all 6; 2 of them are hidden until then.

  1. As given. The problem’s own traffic, scheduled failures and seed.
  2. Peak hour. 25% more traffic than the problem states, with the same failures.
  3. A slow dependency. The main data store or model answers twice as slowly for part of the run.
  4. A server dies. One replica of the service with the most replicas is killed for part of the run.

The traffic

1 requests a second for 45 seconds: 100% question (reads).

The system

What each component does in this design, and what calls it. The simulator mimics each one from these settings.

ComponentIts job hereSettings
API gatewayAPI gatewayTraffic enters here.Calls Research API.Gateways centralise auth, rate limits and routing so each service does not reimplement them.100000 slots · 60,000 calls/min
Research APIApp serviceAPI gateway forwards to it.Calls Lead agent.Little's law: throughput ≈ concurrency ÷ latency. 200 slots at 50 ms serve about 4,000 req/s.1 replicas × 500 slots · timeout 240 s
Lead agentAgent orchestratorResearch API calls it.Calls Model, Subagent pool, Context compactor, Research memory, Checkpoint store.An agent loops: think, call a tool, read the result, repeat until done.62 slots · 2 steps
Subagent poolSubagent poolLead agent calls it.Calls Model, Web search tool, Browser tool.Multi-agent research systems fan a task out to parallel subagents and use many times more tokens than a single agent.250 slots · 4 steps
ModelLLM endpointLead agent calls it. Subagent pool calls it.Latency has two parts: time to the first token, then a steady stream of tokens per second.250 streams · 7,500 calls/min · 400M input tokens/min
Web search toolHybrid searchSubagent pool calls it.Hybrid search combines keyword matching (BM25) with vector similarity.50 slots · 2 steps
Browser toolBrowser toolSubagent pool calls it on 50% of requests.Browser tools let agents read and act on web pages, which is slow and brittle.25 slots
Context compactorContext compactorLead agent calls it on 30% of requests.Compaction summarises a long context so the agent can keep going without hitting the window limit.25 slots
Research memoryAgent memory storeLead agent calls it.Agent memory keeps facts across sessions so the agent does not start from scratch.125 slots
Checkpoint storeCheckpoint storeLead agent calls it.Durable execution saves state after each step so a crash resumes instead of starting over.250 slots

More real world problems