Kubernetes requests and limits: why a Pod at 40% CPU still gets throttled Intermediate

The problem

checkout-api is a NestJS service on Node 24, running 6 replicas. Each container has cpu: 1 and memory: 768Mi as both request and limit. During a promotion, p99 of POST /checkout jumps from 180 ms to 1.2 s.

The dashboard shows each Pod at 40% of its CPU limit. Nothing is OOMKilled, the HPA does not scale, and the database is quiet. (All numbers in this scenario are illustrative.)

The team’s question: how can a Pod that uses less than half of its CPU still be starved for CPU, and which knob should change?

What to set? (summary first)

Situation	Set	Condition
Any production workload	CPU and memory requests from measured usage	Requests are what the scheduler and HPA see
Memory	limit = request	Memory cannot be throttled; overcommit ends in OOM kills or eviction
Latency-sensitive API with bursty CPU (hashing, gzip, JSON)	CPU limit well above request, or no CPU limit	Only if the cluster allows it and throttling is monitored
Strict isolation, benchmarks, exclusive cores	CPU limit = request (Guaranteed)	Accept throttling; size the limit for peak bursts
Node.js in a container	--max-old-space-size ≈ 75% of the memory limit	Leaves room for Buffers, native memory and threads
HPA on CPU	Target utilization relative to request	Every container in the Pod has a CPU request

Core concepts

	Request	Limit
Used by	kube-scheduler, HPA, kubelet eviction ranking	kubelet and container runtime, enforced by the kernel through cgroups
CPU	Relative weight when the node is contended	Hard quota per period: throttling
Memory	Scheduling; hint for memory.min/memory.low on cgroup v2	Above it, the kernel OOM-kills a process in the container
If missing	Defaults to the limit when only a limit is set	No ceiling: the container can use free node capacity

The scheduler places a Pod only where node allocatable minus existing requests fits its requests. Limits are enforced later by the kernel, whatever the scheduler thought.

The combination decides the Pod’s QoS class, which matters when memory runs out:

Loading diagram…

checkout-api with cpu: 1 and memory: 768Mi as both request and limit is Guaranteed. Guaranteed does not mean “never throttled”; it means the CPU limit equals the request.

How it works

CPU limit = CFS quota per 100 ms

The kubelet and container runtime turn a CPU limit into a CFS quota: quota = limit × period, default period 100 ms. cpu: 1 becomes cpu.max = 100000 100000 on cgroup v2: 100 ms of CPU time per 100 ms, summed across all threads of the container.

A Node process is not single-threaded. Async crypto, zlib and fs calls run on the libuv threadpool, which has 4 threads by default. Four parallel hashes can spend the whole quota in a quarter of a period:

Four threadpool threads spend the 100 ms quota in 25 ms; the whole container waits until the next period

◀ Prev▶ PlayNext ▶↺ RestartStep 1/5

t=0: four checkout requests each start a 40 ms hash on four threadpool threads, running on four cores at once.

State after each step:

1. t=0: quota 100 ms, four hashes running on four cores.
2. t=25 ms: 4 × 25 ms = 100 ms used. Every thread in the cgroup is throttled, each hash still 15 ms short.
3. t=30 ms: GET /cart arrives. The event loop belongs to the same cgroup, so it cannot run either.
4. t=100 ms: the quota refills. Throttled threads resume.
5. t=115 ms: hashes finish. Latency is 115 ms instead of 40 ms, yet CPU averaged over one second is 16% of the limit.

This is why “40% of limit” proves nothing. Dashboards average CPU over 30 s to 5 min; throttling happens inside 100 ms. The metric that sees it is container_cpu_cfs_throttled_periods_total divided by container_cpu_cfs_periods_total.

Memory limit: OOMKilled versus eviction

Memory has two independent failure paths:

Exceeding your own limit restarts one container; node pressure evicts whole Pods that use more than their request

Enlarge diagram

checkout-api → cgroup memory.maxallocate

◀ Prev▶ PlayNext ▶↺ RestartStep 1/6

Path 1: the V8 heap and Buffers of checkout-api grow; working set reaches 768Mi.

	OOMKilled	Node-pressure eviction
Trigger	Container exceeds its own memory limit	Node signal such as memory.available<100Mi (default hard threshold on Linux)
Who acts	Kernel OOM killer inside the cgroup	kubelet, before the node runs out
Result	Container restarts in the same Pod, exit 137	Pod phase Failed; the controller creates a new Pod
Victim order	The container that hit its limit	Pods whose usage exceeds requests first, then by priority, then by how far usage exceeds the request

In practice BestEffort Pods (request 0) go first and Guaranteed Pods last, because a Guaranteed Pod cannot use more than its request. If the node hits a real OOM before the kubelet reacts, the kernel picks using oom_score_adj: −997 for Guaranteed, 1000 for BestEffort.

Operational details

Node.js inside a container (measured with Node 24.22 in Docker):

• os.availableParallelism() returns 1 with --cpus=1, but os.cpus().length still shows every host core. The threadpool stays at 4 threads regardless of the CPU limit.
• UV_THREADPOOL_SIZE must be set before the process starts (environment variable), not from code.
• Without flags, V8’s heap limit follows the container limit: 432 MB at 768Mi, 560 MB at 1Gi. The rest of RSS (Buffers, native code, thread stacks) is outside the heap.
• --max-old-space-size=576 at 768Mi gives a 624 MB heap limit (old space plus young generation), so Node fails with JavaScript heap out of memory and a GC trace before the kernel sends a silent SIGKILL.

Sizing from metrics (at least one week, including the busiest day):

• CPU request ≈ p90–p95 of rate(container_cpu_usage_seconds_total[5m]) per Pod at peak.
• Memory request = limit ≈ max container_memory_working_set_bytes plus 20–30% headroom.
• Watch the throttling ratio after every change; a target below 5% of periods is a common starting point, not a standard.

HPA on CPU:

• desired = ceil(current × currentUtilization / target), where utilization is usage divided by request, averaged across Pods.
• Syncs every 15 s, ignores changes within a 10% tolerance, waits 5 minutes before scaling down.
• If any container lacks a CPU request, the HPA takes no action for that metric.

The CPU-limit debate:

• No CPU limit: the container uses idle node CPU and is weighted by its request under contention. No throttling, less predictable latency.
• CPU limit: predictable behaviour, but bursts pay throttling.
• Memory is different: it cannot be reclaimed by slowing a process down, so keep its limit.

Real-world example

checkout-api: throttled at 40% average CPU

During the promotion, the throttled-periods ratio of checkout-api reads 38%. Guest checkout creates an account and hashes the password with the native bcrypt package, which runs on the threadpool; gzip of large cart responses adds more.

The team raises the CPU limit to 2 while keeping the request at 1 (now Burstable), sets UV_THREADPOOL_SIZE=2 to cap parallel hashing, and checks that nodes have allocatable CPU to spare. The throttling ratio drops below 2% and p99 returns to about 200 ms.

The demo runs 20 bursts of four parallel pbkdf2 hashes, 900 ms apart, and reads the cgroup’s cpu.stat:

docker run --rm --cpus=1 -v "$PWD/examples/src/kubernetes-resources-limits:/demo" \
  node:24-alpine node /demo/throttle-demo.ts

Docker CPU limit	p50	p99	Avg CPU	Throttled periods
--cpus=1	105–189 ms	221–299 ms	19–22%	31–39 of 72–81
--cpus=1, UV_THREADPOOL_SIZE=1	78 ms	180 ms	12%	3 of 66
--cpus=2	53 ms	171 ms	23%	12 of 57
--cpus=4	51 ms	83 ms	23%	0 of 53
no limit	55 ms	82 ms	23%	—

Measured on Docker Desktop (cgroup v2, 10 cores); numbers vary between runs. Average CPU stays under a quarter of a core, yet with --cpus=1 about half the active periods are throttled.

Trade-offs

Tight CPU limits (limit = request)

• Predictable performance; tests on staging match production.
• A runaway Pod cannot take CPU from its neighbours.
• Guaranteed QoS; required for exclusive cores with the static CPU manager policy.

Costs

• Short bursts are throttled even when the node has idle cores.
• Latency spikes are invisible on average-CPU dashboards.
• More replicas are needed to absorb bursts, which costs money.

🎮 Try it

• CPU throttling: p99 jumps to 1.2 s while CPU sits at 40% of the limit. Find the metric and the fix.
• Node.js memory leak: a Pod OOMKilled every 6 hours.

Production checklist

1. Every container sets CPU and memory requests derived from at least a week of metrics.
2. Memory limit equals memory request; --max-old-space-size sits at about 75% of it.
3. If CPU limits are kept, alert when the throttling ratio stays above 5% for 10 minutes.
4. UV_THREADPOOL_SIZE is chosen from the CPU limit and the amount of hashing, gzip and fs work.
5. HPA targets are tuned against requests, and every container in the Pod has a CPU request.
6. A load test with realistic bursts runs before changing limits, measuring p99 and throttling together.
7. Node allocatable and the sum of requests per node are checked before raising requests or limits.
8. Alerts exist for OOMKilled restarts and for evicted Pods.

Common mistakes

Common pitfall

Reading average CPU to rule out a CPU problem. A 5-minute average hides 100 ms quota exhaustion. Check container_cpu_cfs_throttled_periods_total before concluding the service is not CPU-bound.

Removing the memory limit to “stop OOMKills” moves the risk to the node:

Memory without a limit

✕ Common

resources:
  requests: { cpu: 500m, memory: 256Mi }   # real usage 700Mi, no limit

✓ Prefer

resources:
  requests: { cpu: "1", memory: 768Mi }
  limits:   { cpu: "2", memory: 768Mi }    # memory limit = request

Why

Usage far above request makes this Pod the first eviction candidate under node pressure, and its growth can push other Pods out.

Misconception	Consequence	Do instead
Guaranteed QoS means no throttling	Bursts still hit the CPU quota	Watch the throttling ratio for every class
HPA will scale when the Pod is throttled	Average usage stays below the target, so the HPA does nothing	Size requests from usage; alert on throttling
More replicas fix throttling	Each Pod still bursts within its own quota	Raise the limit or reduce parallel bursts

Self-check

1. Why can a container with cpu: 1 be throttled while averaging 0.2 cores?
2. A Pod is OOMKilled but the node has free memory. Which limit was hit, and what happens to the Pod?
3. Which Pod does the kubelet evict first under memory pressure, and why is a Guaranteed Pod last?
4. A request of 200m with real usage of 600m: what does the HPA see?

Summary

Key takeaways

• Requests drive scheduling, HPA and eviction ranking; limits are enforced by the kernel.
• A CPU limit is a quota per 100 ms across all threads; bursts get throttled while average CPU looks low.
• Memory over its limit means OOMKilled (137); node pressure evicts Pods using more than their request.
• Keep memory limit = request; treat CPU limits as a trade-off and monitor throttled periods.
• In Node, the threadpool (4 threads) and non-heap memory both count against the container.

References

• Resource Management for Pods and Containers — Kubernetes: how requests and limits are applied and enforced.
• Pod Quality of Service Classes — Kubernetes: exact rules for Guaranteed, Burstable and BestEffort.
• Node-pressure Eviction — Kubernetes: thresholds, eviction ranking and oom_score_adj.
• Horizontal Pod Autoscaling — Kubernetes: the scaling formula, tolerance and stabilization.
• CFS Bandwidth Control — Linux kernel: quota, period and throttling statistics.
• Command-line options — Node.js v24: UV_THREADPOOL_SIZE and heap flags.
• Prometheus metrics — cAdvisor: names of the CFS throttling counters.