Week 12: Kubernetes in Production

1. Ingress & Load Balancing

A ClusterIP Service (the type you used in Week 11) is only reachable from inside the cluster. To let real users hit your app, you need something that terminates external traffic and routes it in — an Ingress does that for HTTP/HTTPS, with host- and path-based routing rules, from a single external entry point instead of one load balancer per Service.

An Ingress resource is just routing rules; it does nothing on its own. You need an Ingress controller — commonly NGINX Ingress Controller or, in a managed cloud cluster, a cloud-provider-native one — running in the cluster to actually read those rules and provision the load balancer.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.1/deploy/static/provider/kind/deploy.yaml
kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=90s

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - host: app.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-app
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: web-api
                port:
                  number: 8080

Notice the shape: the Ingress never talks to Pods directly, only to Services — it sits one more layer above the Service, the same way the Service sits above Pods. Each layer exists to hide instability in the layer below it: Pod IPs are unstable, so Services hide that; the set of Services and their internal ports shouldn't matter to an end user, so Ingress hides that behind one hostname.

2. Horizontal Pod Autoscaling

A fixed replicas: 3 is a guess. The Horizontal Pod Autoscaler (HPA) watches a metric — CPU utilization by default — and adds or removes Pods to keep it near a target, within a min/max bound you set. It needs the metrics-server add-on running in the cluster to see actual usage, and it needs your Deployment to have resources.requests set, because "80% CPU" is meaningless without a requested baseline to measure against.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

kubectl apply -f hpa.yaml
kubectl get hpa web-app --watch
# NAME      REFERENCE            TARGETS   MINPODS   MAXPODS   REPLICAS
# web-app   Deployment/web-app   42%/70%   3         10        3

The HPA scales up quickly (within seconds of a metric breach) but scales down slowly by design — a default 5-minute stabilization window prevents it from flapping Pods up and down every time traffic dips briefly. You can tune that window explicitly with a behavior block if the defaults are too conservative or too jumpy for your traffic pattern.

3. Packaging an App as a Helm Chart

By now you have five or six YAML files per app — Deployment, Service, Ingress, HPA, ConfigMap — and every environment (dev, staging, prod) needs slightly different values for each. Copy-pasting and hand-editing those files per environment is exactly the kind of repetition Terraform modules solved for infrastructure in Week 10. Helm is Kubernetes' package manager: it templates your manifests with a values file per environment, and tracks releases so you can upgrade or roll back as one unit.

helm create web-app
# web-app/
#   Chart.yaml         -- chart name, version
#   values.yaml         -- default configuration
#   templates/
#     deployment.yaml
#     service.yaml
#     ingress.yaml
#     hpa.yaml
#     _helpers.tpl       -- reusable template snippets

replicaCount: 3

image:
  repository: ghcr.io/acme/web-app
  tag: "1.4.0"

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 256Mi

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

A production chart usually gets one values file per environment — values-staging.yaml, values-prod.yaml — overriding just the fields that differ, like replica counts or resource limits, while everything else stays in the shared default.

helm install web-app ./web-app -f values-staging.yaml
helm upgrade web-app ./web-app -f values-prod.yaml --set image.tag=1.5.0
helm list
helm rollback web-app 1   # revert to release revision 1
helm uninstall web-app

4. Rolling Updates, Probes & Self-Healing

By default, updating a Deployment's image triggers a rolling update: new Pods come up gradually while old ones terminate, so there's no moment where zero Pods are serving traffic. Two fields control how cautious that rollout is:

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0   # never drop below the desired replica count
      maxSurge: 1          # allow one extra Pod above desired count during rollout

None of that matters, though, if Kubernetes can't tell whether a new Pod is actually healthy — which is exactly what liveness and readiness probes are for. A liveness probe that fails causes Kubernetes to restart the container; a readiness probe that fails just removes the Pod from the Service's load-balancing pool without restarting it, which matters a lot for an app that's still starting up or briefly overloaded.

containers:
  - name: web
    image: ghcr.io/acme/web-app:1.4.0
    ports:
      - containerPort: 8080
    readinessProbe:
      httpGet:
        path: /healthz/ready
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 10
    livenessProbe:
      httpGet:
        path: /healthz/live
        port: 8080
      initialDelaySeconds: 15
      periodSeconds: 20
      failureThreshold: 3

Combine a rolling update with readiness probes and Kubernetes will hold the rollout open — old Pods keep serving — until each new Pod reports ready, giving you an automatic health gate on every deploy without writing any extra pipeline logic.

When a container keeps failing its liveness probe or exiting non-zero, Kubernetes restarts it with an exponential backoff — 10s, 20s, 40s, capping at 5 minutes — and reports the Pod status as CrashLoopBackOff once it's been restarted repeatedly. That status is a symptom, never the root cause.

kubectl get pods
# web-app-6f9c9d5b7-k2n4x   0/1   CrashLoopBackOff   6   14m

kubectl describe pod web-app-6f9c9d5b7-k2n4x   # Events: OOMKilled? liveness probe failing?
kubectl logs web-app-6f9c9d5b7-k2n4x            # what the current container printed
kubectl logs web-app-6f9c9d5b7-k2n4x --previous # logs from the container BEFORE the last restart

5. Hands-on Exercise

6. Knowledge Check

Four quick questions. Expand each to check your answer.