Week 6: Caching Strategies: Cache-Aside, Write-Through & Write-Behind

1. Where a Cache Fits in the Request Path

A cache is a small, fast store that holds a copy of data whose source of truth lives somewhere slower — typically the database. Recall Week 1's latency numbers: a main memory reference is roughly 100 ns, while a disk seek is 2–10 ms — five orders of magnitude slower. A cache exploits that gap by keeping frequently accessed data in memory, so most requests never touch the disk-backed database at all.

[Client] --> [Load Balancer] --> [App Server] --> [Cache] --(miss)--> [Database]
                                                       |
                                                  (hit) returns immediately,
                                                  database never touched

cache hit ratio = hits / (hits + misses)

Caching only pays off when access has locality — a small subset of data accounts for most requests, often approximated by the 80/20 rule (roughly 20% of keys drive 80% of traffic). A perfectly uniform access pattern, where every key is equally likely to be requested, gets little benefit from caching no matter how big the cache is, because the cache can't hold everything and misses stay high regardless of size. The cache hit ratio — the fraction of requests served from cache — is the single most important metric for judging whether a cache is earning its complexity.

2. Cache-Aside (Lazy Loading)

Cache-aside — also called lazy loading — is the most common caching pattern in interview answers and in production systems alike. The application, not the cache, is responsible for keeping the cache populated: on a read, the app checks the cache first; on a hit, it returns the cached value; on a miss, it reads from the database, writes that value into the cache, and then returns it. Writes typically go straight to the database, with the corresponding cache entry either invalidated (deleted) or left to expire.

function getUser(id):
    value = cache.get(id)
    if value is not null:
        return value          # cache hit
    value = db.query("SELECT * FROM users WHERE id = ?", id)
    cache.set(id, value, ttl=300)   # populate on miss
    return value

function updateUser(id, newData):
    db.update("users", id, newData)
    cache.delete(id)           # invalidate, don't update in place

Cache-aside's strengths make it the default choice: only data that's actually requested ever gets cached (no wasted memory on cold data), and if the cache goes down entirely, the system degrades to hitting the database directly on every request — slower, but still correct and available. Its weaknesses are the first request for any given key always misses (a "cold" cache, or cold start after a deploy or eviction), and there's a brief window after a write where a concurrent read can still see the old cached value until invalidation completes — a race condition Week 7 examines in more depth alongside the related "thundering herd" problem, where many concurrent misses on the same key all hit the database at once.

3. Write-Through & Write-Behind (Write-Back)

Cache-aside treats the cache as populated only by reads. The other two strategies involve the cache directly in the write path. In write-through caching, every write goes to the cache and the database together, synchronously, before the write is acknowledged to the client — the cache is always consistent with the database because it's updated as part of every write, not just reads. Write-behind (or write-back) caching writes to the cache immediately and acknowledges the client right away, then asynchronously flushes the write to the database sometime later, often batched.

# Write-through: write is not "done" until both succeed
function updateUser_writeThrough(id, newData):
    cache.set(id, newData)
    db.update("users", id, newData)   # synchronous, on critical path
    return success

# Write-behind: write returns fast, DB write happens later
function updateUser_writeBehind(id, newData):
    cache.set(id, newData)
    writeQueue.enqueue({id, newData})  # async flush, batched
    return success   # client sees this before the DB is touched

# A background worker later drains writeQueue and applies
# batched updates to the database.

The three strategies trade off differently along the same axes:

Cache-aside:
  Write latency:  fastest (cache untouched on write)
  Read-after-write risk: brief staleness until invalidation lands
  Data-loss risk: none (DB is always the write target)

Write-through:
  Write latency:  slower (pays cache + DB synchronously)
  Read-after-write: always consistent, cache is never stale
  Data-loss risk: none (DB write is synchronous)

Write-behind:
  Write latency:  fastest (only cache write is on critical path)
  Read-after-write: consistent from cache, but DB briefly lags
  Data-loss risk: real -- a cache crash before flush loses writes

Write-behind's low write latency and high write throughput make it attractive for workloads like view counters, activity logs or analytics events, where losing a small amount of recent data on a crash is an acceptable cost — an explicit choice to favor availability and throughput over strict durability, the same kind of tradeoff Week 9's CAP discussion formalizes. Because the flush is asynchronous and typically batched, a durable queue between the cache and the database (Week 8) is what makes write-behind safe enough for production rather than a silent data-loss risk.

4. Local vs. Distributed Caching

A local (in-process) cache lives in the same process as the application — a plain in-memory map, or a library like Caffeine or Guava Cache. It's the fastest option because there's no network hop at all, but it has real limits: its capacity is bounded by that one instance's memory, its contents vanish on restart or deploy, and — critically in a horizontally scaled service — every app server has its own independent copy, so the same key can be cached with different values (or cached on one server and not another) across the fleet.

A distributed cache — Redis or Memcached, typically run as its own cluster — is shared across every app server. All instances see the same cached value for a given key, it survives any single app server restarting, and it can itself be sharded (Week 5's sharding concepts apply here too) to scale beyond one cache node's memory. The cost is a network round trip on every access — still vastly faster than a database round trip, but slower than an in-process lookup.

Local (in-process), per app server:
[App Server 1: local cache]   [App Server 2: local cache]
      (independent, can disagree on the same key)

Distributed, shared cluster:
[App Server 1] --\
[App Server 2] ---> [Redis / Memcached cluster] --> [Database]
[App Server 3] --/
      (one shared view of every cached key)

Hybrid (common in high-throughput systems):
[App Server] --> L1: local cache (nanoseconds)
                 --> L2: distributed cache (sub-millisecond)
                 --> L3: database (milliseconds)

The deciding question is whether the application can tolerate different servers seeing different cached values for the same key. A read-mostly, slowly changing reference dataset (feature flags, a small config table) is often fine as a local cache — the inconsistency window between instances is small and low-stakes. Anything where correctness depends on every server agreeing — session data, inventory counts, rate-limit counters (Week 11) — needs a distributed cache as the single shared source of cached truth.

5. Hands-on Exercise

6. Knowledge Check

Four quick questions. Expand each to check your answer.