Skip to content

Scale Lab: build a system, run traffic, watch it fall over Basic

Estimated reading time: 4 min read

Your shop grows round by round. Drag components onto the board, wire the boxes together, then press ▶ Run.

  • 8 rounds, from your first app to a flash sale and a Redis restart at peak hour.
  • Pass a round by answering ≥ 99% of requests with average response time within the limit and cost within budget. The cheapest design earns 3 stars.
  • What if it breaks? The failed box is circled in red, and the feedback says why. Click a feedback item to jump to that box.
  • Wiring: drag from one box’s dot to another box, or click a box, then 🔗 Connect to another box and tap a highlighted box. Removing a wire: click the wire, then ✕, or use the Remove button in the box’s panel.
  • On phones: the flow runs top to bottom; tap a component to add it, tap a box to configure it.

Finished all 8 rounds? Try the multi-round interview problems (your design carries over between rounds):

Problem Difficulty Main pressure
URL shortener Medium Read-heavy, viral links, spam bots
Notifications Medium Slow and failing provider, retries, duplicate sends
Concert ticketing Hard One hot inventory row, row locks, scalper bots
Realtime chat Hard Polling or WebSocket, Pub/Sub between gateways, reconnect storm

Or set your own traffic and incidents in the Sandbox. The numbers in the game are round assumptions meant to show trends; see the model assumptions.

Component Capacity per instance (S / M / L) Price/month
Server 500 / 1,000 / 2,000 req/s $15 / $30 / $60
Database, Read Replica 500 / 1,000 / 2,000 query/s $15 / $30 / $60
Cache 10k / 25k / 50k ops/s, 90% hit rate when warm $20 / $40 / $80
CDN absorbs 95% of static file requests $20 / $40 / $80
Load Balancer, Rate Limiter LB 10k / 25k / 50k; Rate Limiter 20k / 50k / 100k req/s $10 / $20 / $40
Queue / Worker Worker 100 / 200 / 400 job/s $10 / $20 / $40
WebSocket Gateway 10k / 20k / 40k connections $15 / $30 / $60
Pub/Sub 50k / 100k / 200k msg/s $20 / $40 / $80
  • Every dynamic request goes through a server and produces one query: reads go to the cache (if any) then the database/replica, writes go to the primary.
  • Service time grows with utilization ρ = load / capacity, queueing-style: base / (1 − ρ). Above 100%, the excess is dropped.
  • Adding server replicas only helps when a Load Balancer sits in front; the Load Balancer takes dead replicas out of rotation.
  • With request coalescing enabled on the cache, only ~5% of the extra misses reach the database when the cache is empty; the rest wait for the result of the in-flight query.
  • The Rate Limiter returns 429 to bots at the edge; real users’ requests pass through normally.
  • Hot row (ticketing problem): a row lets only one transaction hold its lock, so it can be decremented at most 1000 / lock hold time (ms) times per second, multiplied by the number of rows if inventory is split.
  • Lock hold time (ticketing problem): calling payment inside the transaction ~500 ms, FOR UPDATE ~20 ms, a single UPDATE statement ~2 ms.
  • Chat: without a gateway, clients poll the server every 2 seconds (messages arrive 1 second late on average). With 2 or more gateway instances but no Pub/Sub, messages to recipients on another gateway are lost.
  • Chat reconnects: when a gateway dies, its clients reconnect, and every reconnect is an authentication request. Without jitter they all land in the same second; with jitter they spread over ~20 seconds.
  • Provider outage (notifications problem): every send fails. Without retries the message is dropped; with retries it goes back to the queue, and ~5% of the “failed” sends actually arrived, so they are sent twice unless the worker deduplicates.

This model is deliberately simple to show the right trends (bottlenecks, the cost of each choice), not to predict real numbers. Real systems need measurement: see SLI, SLO and error budget.