At a glance
- Did you know
- Multi-agent research systems fan a task out to parallel subagents and use many times more tokens than a single agent.
- Rule of thumb
- Use them for broad, parallel work; give each subagent a clear goal, tools and output format.
- Watch out
- Token use multiplies with subagents; tools saturate under the fan-out.
Its settings in ScaleMySystems
What you can change on it in the lab, and what each number does.
- Concurrency (slots)
- Calls it can work on at once (per instance when autoscaling). Needed slots ≈ calls per second × seconds each takes. Extra calls wait in the queue, then are rejected.
- Queue capacity
- Calls allowed to wait for a free slot. A bigger queue absorbs bursts but adds waiting time; 0 rejects straight away.
- vCPU
- CPU cores available. Needed ≈ calls per second × CPU per request ÷ 1000. Above about 90% busy it slows down and may crash.
- CPU per request (ms)
- CPU time each call burns. Part of what the component is; make the code cheaper to lower it.
- Latency (p50) (ms)
- Typical time one call takes here, before any calls it makes itself. Slots are held for this long.
- Latency (p99) (ms)
- The slowest 1% of calls. Callers must set their timeout above this or they cut off healthy calls.
- Timeout to dependencies (ms)
- How long it waits for each call it makes before giving up. Set it above the dependency’s p99 but low enough that a dead dependency does not hold every slot.
- Error rate
- Share of calls that fail on their own (bugs, declines, flaky tools). An agent that calls it on every step multiplies this.
- Quota (req/min)
- Calls allowed per minute (an API key or provider limit). Calls over it get 429s. Needed = calls per second × 60, with headroom.
- Calls per request (fan-out)
- How many times its outgoing calls run per request: agent steps, jobs, subagents. Load on everything behind it is multiplied by this, and any step failing fails the request.
Problems that start with it
- Subagents swamp the search tool A lead agent fans each question out to 8 parallel subagents, and each subagent searches the web. The search tool has 8 slots.
- Multi-agent token bill Every subagent sends 3,000 tokens of context. Eight subagents per question multiply token use past the provider quota.
- Fragile subagents Eight subagents each call a tool that fails 1% of the time; any subagent failure fails the question.
- Research agents share few browsers A deep-research system fans out to 5 subagents that each browse the web; the browser pool has 16 browsers.
- Deep research: from launch build to real traffic A multi-agent research product on its first servers: the lead agent fans each question out to sub-agents that search and browse the web, and the browser tool, sub-agent pool and model capacity are an eighth of their real size. Questions arrive one a second and each takes minutes. One in ten fails.
Full lesson coming soon
Lessons are being added one component at a time: how it works, its settings, a live exercise to break and fix it, interview questions and a short check. See the Circuit breaker lesson for what one looks like.