Problems / Real world, medium
A coding agent platform: each task is a loop of model calls, tool calls, code runs in a sandbox and checkpoints, minutes long. Twenty seconds in, the model provider has an outage for 30 seconds. Every step that needs the model fails, and with it the whole task: two in three tasks are lost.
Unlock it once for 15 credits; everything inside it is free (runs, changes, Submit, results, hints). Sign in with Google to start; new accounts get free credits.
Keep at least 95% of tasks succeeding through the outage, with p99 under 3 minutes.
Like a coding problem, a solution has to pass every case, not just the one it was built for. Submit in the lab runs all 5; 2 of them are hidden until then.
2 requests a second for 45 seconds: 100% task (writes).
What each component does in this design, and what calls it. The simulator mimics each one from these settings.
| Component | Its job here | Settings |
|---|---|---|
| API gatewayAPI gateway | Traffic enters here.Calls Agent API.Gateways centralise auth, rate limits and routing so each service does not reimplement them. | 100000 slots · 60,000 calls/min |
| Agent APIApp service | API gateway forwards to it.Calls Agent harness.Little's law: throughput ≈ concurrency ÷ latency. 200 slots at 50 ms serve about 4,000 req/s. | 3 replicas × 500 slots · timeout 240 s |
| Agent harnessAgent harness | Agent API calls it.Calls Model (primary provider), Tool policy engine, Tool servers (files, grep, git), Code sandboxes, Context compactor, Checkpoint store, Tracing.A harness is everything around the model in an agent: context management, tools, checkpoints, retries and evaluation. | 1000 slots · 8 steps |
| Model (primary provider)LLM endpoint | Agent harness calls it.Latency has two parts: time to the first token, then a steady stream of tokens per second. | 600 streams · 20,000 calls/min · 200M input tokens/min |
| Tool policy engineTool policy engine | Agent harness calls it.A policy engine checks each tool call against what this agent, user and data may do. | 4000 slots |
| Tool servers (files, grep, git)Tool / MCP server | Agent harness calls it.Tools give the model actions such as search or database queries; each call is another round trip. | 2000 slots |
| Code sandboxesCode sandbox | Agent harness calls it on 40% of requests.Sandboxes run model-written code in an isolated container with limited network and time. | 400 slots |
| Context compactorContext compactor | Agent harness calls it on 15% of requests.Compaction summarises a long context so the agent can keep going without hitting the window limit. | 200 slots |
| Checkpoint storeCheckpoint store | Agent harness calls it.Durable execution saves state after each step so a crash resumes instead of starting over. | 2000 slots |
| TracingAgent tracing | Agent harness calls it.Agent traces record every model and tool call with inputs, outputs, tokens and timing. | 5000 slots · 6,000,000 calls/min |