Requests, Limits, and Why Your Pod Got OOMKilled

Two containers land on the same node. One declares nothing, the other declares requests and limits. The node starts running out of memory — and it's the undeclared container that dies first. Understanding why is the difference between a workload that degrades gracefully and one that produces pagers.

Requests are for the scheduler. When it places a pod, it sums each container's requests.cpu and requests.memory and looks for a node with enough unallocated headroom. Requests are also your guaranteed share: under contention, the kernel weights CPU time by requests.

Limits are for the kernel. CPU is compressible — exceed limits.cpu and the container gets throttled, not killed. Memory is not compressible — exceed limits.memory and the OOM killer terminates a process. You'll see OOMKilled and exit code 137 in kubectl describe pod.

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: "1"
    memory: 512Mi

The relationship between the two sets your QoS class, which determines eviction order when a node is under pressure:

• Guaranteed: requests == limits for every container. Last to be evicted.
• Burstable: requests set but lower than limits. Middle of the pack.
• BestEffort: no requests at all. First to die, every time.

See reality with kubectl top pods (needs metrics-server) and kubectl describe node <name> — the Allocated resources section shows how full the node is. To protect yourself from developers who forget, add a LimitRange to the namespace to inject default requests/limits, and a ResourceQuota to cap what the namespace can consume in total.

A modern JVM reads cgroup limits and sizes its heap accordingly (this has been default since JDK 8u191 / JDK 10+), but older runtimes and runtimes like Go need care: Go will happily set GOMAXPROCS to the node's core count and spawn threads your CPU limit can't feed. Set GOMEMLIMIT or use automaxprocs on Go services.

Try this

Deploy a pod with requests.memory: 32Mi and limits.memory: 64Mi running stress-ng --vm 1 --vm-bytes 200M. Watch it OOMKill, then inspect kubectl get pod -o jsonpath='{.status.containerStatuses[0].lastState}' to see the exit reason.

Pitfalls

• Pods with no requests schedule anywhere and evict first. Any pod that matters needs requests.
• Aggressive CPU limits cause latency spikes that look like application bugs — throttling shows up as nr_throttled in the cgroup stats, not in your logs. Many shops set CPU requests but no CPU limit for latency-sensitive services.
• Requests sized for a "typical" node can make pods unschedulable on your actual node sizes. Pending pods with Insufficient cpu in events are usually this, not a cluster outage.