Problems / AI security and governance, medium
An ops copilot reads public issues (untrusted), queries private repos and the customer database (private data), and posts comments, PRs and webhooks (outbound). That is the lethal trifecta. A prompt-injection classifier that stops 97% of attacks guards it, and attackers file 50 poisoned issues a second for 15 seconds.
Unlock it once for 15 credits; everything inside it is free (runs, changes, Submit, results, hints). Sign in with Google to start; new accounts get free credits.
No data sent out by an attack, success ≥ 95%.
Like a coding problem, a solution has to pass every case, not just the one it was built for. Submit in the lab runs all 6; 2 of them are hidden until then.
3 requests a second for 45 seconds: 100% task (writes).
What each component does in this design, and what calls it. The simulator mimics each one from these settings.
| Component | Its job here | Settings |
|---|---|---|
| Prompt-injection classifierPrompt-injection filter | Traffic enters here.Calls Load balancer.Prompt injection hides instructions in input or retrieved content; no filter catches all of them. | 100000 slots |
| Load balancerLoad balancer | Prompt-injection classifier forwards to it.Calls Agent API.Layer 4 balancers route by IP and port; layer 7 balancers read HTTP and can route by path or header. | – |
| Agent APIApp service | Load balancer forwards to it.Calls Agent orchestrator.Little's law: throughput ≈ concurrency ÷ latency. 200 slots at 50 ms serve about 4,000 req/s. | 3 replicas × 500 slots · timeout 180 s |
| Agent orchestratorAgent orchestrator | Agent API calls it.Calls Model, Private repos and customer DB (MCP), Outbound: comments, PRs, webhooks.An agent loops: think, call a tool, read the result, repeat until done. | 400 slots · 4 steps |
| Outbound: comments, PRs, webhooksExternal API (mock) | Agent orchestrator calls it on 20% of requests.Third-party APIs have their own rate limits and outages that you cannot fix. | 2000 slots · 60,000 calls/min |
| ModelLLM endpoint | Agent orchestrator calls it.Latency has two parts: time to the first token, then a steady stream of tokens per second. | 2000 streams |
| Private repos and customer DB (MCP)Tool / MCP server | Agent orchestrator calls it.Tools give the model actions such as search or database queries; each call is another round trip. | 2000 slots |