Problems / Cloud and cost, medium

Java cold starts at launch

A Java Lambda (Spring) takes 3 s to initialise. At 30 s a launch email quintuples traffic for 20 s: every request without a warm environment starts a new one and waits for INIT, which needs still more environments.

Unlock it once for 15 credits; everything inside it is free (runs, changes, Submit, results, hints). Sign in with Google to start; new accounts get free credits.

Your goals

Keep success at 99% or more and p99 under 1 s through the launch.

Test cases

Like a coding problem, a solution has to pass every case, not just the one it was built for. Submit in the lab runs all 3; 1 of them are hidden until then.

  1. As given. The problem’s own traffic, scheduled failures and seed.
  2. A slow dependency. The main data store or model answers twice as slowly for part of the run.

The traffic

100 requests a second for 120 seconds: 100% api (reads).

The system

What each component does in this design, and what calls it. The simulator mimics each one from these settings.

ComponentIts job hereSettings
API Gateway (HTTP API)API gatewayTraffic enters here.Calls Orders API (Java Lambda).Gateways centralise auth, rate limits and routing so each service does not reimplement them.100000 slots · 600,000 calls/min
Orders API (Java Lambda)Serverless functionAPI Gateway (HTTP API) forwards to it.Calls DynamoDB.Serverless functions scale per request, but a cold start adds latency to the first call.1000 slots
DynamoDBKey-value storeOrders API (Java Lambda) calls it.Key-value stores spread keys over partitions; one very popular key can overload a single partition.40 partitions × 3,000 ops/s

More cloud and cost problems