At a glance
- Did you know
- Serverless functions scale per request, but a cold start adds latency to the first call.
- Rule of thumb
- Watch the account concurrency limit: past it, requests are throttled.
- Watch out
- Cold starts on bursts (seconds for Java); 429s at the account's 1,000 concurrency; one function starving the rest; costly at steady load.
Its settings in ScaleMySystems
What you can change on it in the lab, and what each number does.
- Concurrency (slots)
- Calls it can work on at once (per instance when autoscaling). Needed slots ≈ calls per second × seconds each takes. Extra calls wait in the queue, then are rejected.
- Queue capacity
- Calls allowed to wait for a free slot. A bigger queue absorbs bursts but adds waiting time; 0 rejects straight away.
- CPU per request (ms)
- CPU time each call burns. Part of what the component is; make the code cheaper to lower it.
- Latency (p50) (ms)
- Typical time one call takes here, before any calls it makes itself. Slots are held for this long.
- Latency (p99) (ms)
- The slowest 1% of calls. Callers must set their timeout above this or they cut off healthy calls.
- Timeout to dependencies (ms)
- How long it waits for each call it makes before giving up. Set it above the dependency’s p99 but low enough that a dead dependency does not hold every slot.
- Error rate
- Share of calls that fail on their own (bugs, declines, flaky tools). An agent that calls it on every step multiplies this.
- Quota (req/min)
- Calls allowed per minute (an API key or provider limit). Calls over it get 429s. Needed = calls per second × 60, with headroom.
- Memory (MB)
- Lambda memory, which also sets its CPU share. You pay for memory × run time (GB-seconds).
- Reserved concurrency
- Executions kept for this function alone, out of the account's limit (1,000 by default). It is also the function's ceiling. Empty = shares the account pool.
- Provisioned concurrency
- Environments kept initialised all the time: no cold start for them, billed per GB-second whether used or not.
- Cold start (INIT) (ms)
- Time to start a new environment: about 250 ms for Node.js or Python, 3 s for Java, a few hundred ms with SnapStart. Billed since August 2025.
Problems that start with it
- Java cold starts at launch A Java Lambda (Spring) takes 3 s to initialise. At 30 s a launch email quintuples traffic for 20 s: every request without a warm environment starts a new one and waits for INIT, which needs still more environments.
- One function starves the account Two Lambda functions share an account: the customer API (about 100 ms a call) and a report export (about 4.5 s a call). Exports run 240 a second at month end. Neither has reserved concurrency, and the account limit is 1,000.
Full lesson coming soon
Lessons are being added one component at a time: how it works, its settings, a live exercise to break and fix it, interview questions and a short check. See the Circuit breaker lesson for what one looks like.