Week 14: Resource Management, Scaling & the HPA

1. Requests & Limits, Precisely

These are two different mechanisms doing two different jobs, and CPU and memory behave differently once a limit is hit — a distinction most tutorials gloss over.

what each field actually controls

requests   # what the SCHEDULER uses to decide which node has room for this Pod.
           # A Pod won't be scheduled onto a node that can't satisfy its requests.

limits     # a hard ceiling enforced by the kernel (via cgroups — Week 1's primitives,
           # applied by Kubernetes automatically) on how much the container can use.

CPU vs. memory limits — different consequences

resources:
  requests:
    cpu: "250m"      # 0.25 of a CPU core, guaranteed available
    memory: "256Mi"
  limits:
    cpu: "1000m"      # CPU limit exceeded -> the process is THROTTLED, not killed.
                       # It just runs slower; no crash, no restart.
    memory: "512Mi"    # Memory limit exceeded -> the process is OOM-KILLED immediately.
                       # There's no "slow down" option for memory the way there is for CPU.

This asymmetry matters operationally: a CPU limit set too low silently degrades performance (annoying, but survivable). A memory limit set too low causes hard crashes and restarts — kubectl get pods showing OOMKilled in the status is exactly this happening.

Measure before guessing

Setting requests/limits from a guess is how you get either wasted capacity (set too high, nodes look "full" when they're not) or OOM-killed Pods (set too low). Watch kubectl top pod under real or realistic load before finalizing these numbers for anything that matters.

2. QoS Classes & OOM Kills

How you set requests and limits relative to each other assigns your Pod a Quality of Service class, which determines eviction priority when a node runs low on resources.

the three QoS classes

Guaranteed   # requests == limits for EVERY container in the Pod. Last to be
             # evicted under node pressure — the safest tier.

Burstable    # requests set, but lower than limits (the common case). Evicted
             # before Guaranteed Pods, after BestEffort ones, under pressure.

BestEffort   # NO requests or limits set at all. First to be evicted the moment
             # a node runs low on resources — the least protected tier.

checking a Pod's assigned QoS class

kubectl get pod my-app-xyz -o jsonpath='{.status.qosClass}'
kubectl describe pod my-app-xyz | grep -A2 "QoS Class"

This is the concrete answer to "why should I bother setting requests/limits at all?" — a Pod with neither is BestEffort, the first thing evicted the moment a node gets tight on resources, regardless of how important that workload actually is to you.

Guaranteed isn't automatically "best" for everything

Setting requests==limits everywhere gives up the ability to burst above your baseline usage during a traffic spike, and can waste capacity if your real usage varies a lot. Reserve Guaranteed for genuinely critical, steady workloads; Burstable is the right default for most apps.

3. The Horizontal Pod Autoscaler

The HorizontalPodAutoscaler (HPA) watches a metric — most commonly CPU utilization — and adjusts a Deployment's replica count automatically to keep that metric near a target.

hpa.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70    # scale up when average CPU exceeds 70% of REQUESTS

Notice "70% of requests," not "70% of limits" — this is exactly why setting requests correctly (Section 1) isn't optional if you're using an HPA; the target percentage is meaningless without an accurate baseline to measure against.

terminal — watching the HPA work

kubectl apply -f hpa.yaml
kubectl get hpa                          # current replicas, current vs. target CPU%
kubectl get hpa my-app-hpa --watch       # live updates as load changes
kubectl top pods                          # real-time CPU/memory usage per Pod

The HPA needs metrics-server installed

kubectl top and the HPA both depend on the metrics-server add-on being installed in your cluster — if kubectl get hpa shows <unknown> for current CPU usage, that's almost always the missing piece. Minikube ships it as an enableable addon (minikube addons enable metrics-server).

4. Watching It Scale, Live

Theory is convincing; watching replica count climb in real time under generated load is more convincing. A minimal load generator, run against your app's Service:

a simple in-cluster load generator

kubectl run load-generator --image=busybox --restart=Never -it --rm -- \
  /bin/sh -c "while true; do wget -q -O- http://my-app; done"

# In a second terminal, watch the HPA react:
kubectl get hpa my-app-hpa --watch

Within a minute or two of sustained load, you should see the HPA's current CPU% climb, and replicas increase to compensate. Stop the load generator (Ctrl+C, then let the pod clean itself up) and watch replicas scale back down after the HPA's default cooldown window.

Scale-down is deliberately slower than scale-up

The HPA reacts quickly to rising load but has a built-in stabilization window before scaling back down, specifically to avoid "flapping" — rapidly scaling up and down in response to bursty, noisy traffic. This is a deliberate design choice, not a lag bug.

5. Hands-on Exercise

Hands-on

Trigger an OOM kill on purpose, then set up and prove an HPA works

See a memory limit actually kill a container, then build and load-test a real autoscaling setup so both halves of this week are things you've witnessed, not just read.

Part 1 — Cause an OOM kill deliberately:

Deploy any small container with a deliberately tiny memory limit (e.g. limits: memory: "20Mi") running something that allocates more than that — a simple stress --vm 1 --vm-bytes 100M container image works well for this.
Watch kubectl get pods — you should see it restart, with a RESTARTS count climbing.
Run kubectl describe pod <name> and find the "Last State: Terminated" section — confirm the Reason field shows OOMKilled.
Fix it by raising the memory limit to something reasonable, and confirm the restarts stop.

Hint

Kubernetes' default restart policy on a crashed container adds an exponential backoff between restart attempts — if your Pod seems "stuck" for a while between restarts, that's expected behavior (CrashLoopBackOff), not a hang.

Part 2 — Set up and prove an HPA:

Enable metrics-server if it isn't already, and confirm kubectl top pods returns real numbers (not an error).
Deploy an app with sensible CPU requests/limits set, and an HPA targeting 50-70% CPU utilization with min/max replicas of your choice.
Run a load generator against it and watch, in a second terminal, both kubectl top pods and kubectl get hpa --watch as replicas increase.
Stop the load and time how long it takes replicas to scale back down to the minimum — write down what you observed and why the delay is intentional.

Hint

If replicas never scale up, double check your CPU requests aren't set unrealistically high relative to what the container actually uses — a generous request makes 70% utilization very hard to reach even under real load.

6. Knowledge Check

Four quick questions. Expand each to check your answer.

What happens when a container exceeds its CPU limit, vs. exceeding its memory limit?

Exceeding a CPU limit throttles the container — it runs slower, but doesn't crash or restart. Exceeding a memory limit gets the container OOM-killed immediately, since there's no equivalent "slow down" behavior for memory the way there is for CPU time-slicing.

Which QoS class is a Pod assigned if it has no resources.requests or limits set at all?

BestEffort — the least protected tier, first to be evicted the moment a node runs low on resources. This is the concrete cost of skipping requests/limits: no protection at all when a node is under pressure.

An HPA is set to target 70% CPU utilization. Is that 70% of the container's limit or its request?

Its request. This is exactly why an accurate CPU request matters when using an HPA — the utilization percentage is measured against the requested baseline, not the limit, so a poorly-set request makes the target percentage meaningless.

Why does the HPA scale down more slowly than it scales up?

It's a deliberate stabilization window to prevent "flapping" — rapidly scaling replicas up and down in response to bursty or noisy traffic patterns, which would be wasteful and potentially destabilizing. Scaling up reacts quickly to protect availability; scaling down is more conservative.