At a glance
- Did you know
- A harness is everything around the model in an agent: context management, tools, checkpoints, retries and evaluation.
- Rule of thumb
- Externalise progress to durable notes so a fresh session can pick up where the last one stopped.
- Watch out
- Too many turns per task; state lost without checkpoints.
Its settings in ScaleMySystems
What you can change on it in the lab, and what each number does.
- Concurrency (slots)
- Calls it can work on at once (per instance when autoscaling). Needed slots ≈ calls per second × seconds each takes. Extra calls wait in the queue, then are rejected.
- Queue capacity
- Calls allowed to wait for a free slot. A bigger queue absorbs bursts but adds waiting time; 0 rejects straight away.
- vCPU
- CPU cores available. Needed ≈ calls per second × CPU per request ÷ 1000. Above about 90% busy it slows down and may crash.
- CPU per request (ms)
- CPU time each call burns. Part of what the component is; make the code cheaper to lower it.
- Latency (p50) (ms)
- Typical time one call takes here, before any calls it makes itself. Slots are held for this long.
- Latency (p99) (ms)
- The slowest 1% of calls. Callers must set their timeout above this or they cut off healthy calls.
- Timeout to dependencies (ms)
- How long it waits for each call it makes before giving up. Set it above the dependency’s p99 but low enough that a dead dependency does not hold every slot.
- Error rate
- Share of calls that fail on their own (bugs, declines, flaky tools). An agent that calls it on every step multiplies this.
- Quota (req/min)
- Calls allowed per minute (an API key or provider limit). Calls over it get 429s. Needed = calls per second × 60, with headroom.
- Calls per request (fan-out)
- How many times its outgoing calls run per request: agent steps, jobs, subagents. Load on everything behind it is multiplied by this, and any step failing fails the request.
Problems that start with it
- A coding harness that never stops A long-running coding harness takes 20 turns per task, and a 1% tool failure rate sinks a fifth of tasks.
- Harness compaction backs up The harness compacts context on every turn with a summariser that has 4 slots.
- Harness progress notes The harness writes progress notes and a checkpoint after every turn to a store with 6 slots.
- Harness test runs A coding harness runs the test suite in a sandbox every turn. 8 sandboxes, about 1.2 s per run.
- Harness loses its model A long-running harness depends on one model provider, which is down for 20 seconds mid-run.
- Harness traces over quota Every harness turn is traced to an API capped at 600 spans a minute.
- Malicious issue text A coding harness reads issue descriptions. Attackers file issues with instructions to exfiltrate secrets, 100 a second.
- Background agents poll GitHub until throttled Background coding agents open a pull request, then wait for CI by polling GitHub every 10 s, about 12 checks per task. All agents share one GitHub App installation, limited to 15,000 requests an hour (250 a minute). 70 tasks start a minute.
And 1 more on the problems page.
Full lesson coming soon
Lessons are being added one component at a time: how it works, its settings, a live exercise to break and fix it, interview questions and a short check. See the Circuit breaker lesson for what one looks like.