Week 11: Caching & Performance

1. Redis Cache-Aside Pattern

There are two common shapes for putting a cache in front of a database: cache-aside (lazy loading) and write-through. With cache-aside, your application code owns the cache explicitly: on a read, check Redis first; on a miss, read the database and populate the cache; on a write, update the database and then deal with the now-stale cache entry (the next section). With write-through, the cache is updated synchronously as part of every write, so it's never behind the database — at the cost of a Redis round trip on every write, even for rows nobody reads again soon. Cache-aside is the better default for most FastAPI services: it only spends effort caching things that are actually being read, and it degrades gracefully — if Redis is unreachable, cache-aside just falls back to the database, as long as you catch the connection error.

Here's the pattern applied to a task-detail read:

async def get_task(task_id: int) -> TaskOut:
    key = f"task:{task_id}"
    if cached := await redis.get(key):
        return TaskOut.model_validate_json(cached)

    task = await repository.get(task_id)
    if task is None:
        raise TaskNotFoundError(task_id)

    payload = TaskOut.model_validate(task)
    await redis.set(key, payload.model_dump_json(), ex=60)
    return payload

Walking through it: redis.get(key) returns None on a miss and the raw value on a hit, so the walrus operator (:=) lets you check and bind in one expression. On a hit, TaskOut.model_validate_json parses and validates the cached JSON straight into a TaskOut instance — the same work as json.loads() followed by model_validate(), but in one call that Pydantic optimizes specifically for raw JSON text. On a miss, you go to the real repository, convert the ORM row to your response schema with model_validate, and write it back with model_dump_json() — Redis only stores bytes and strings, so a Pydantic model always has to be serialized on the way in and deserialized on the way out.

2. Safe Invalidation & TTLs

The moment you cache something, you've created a second copy of the truth that can drift from the database. The ex=60 TTL bounds how long that drift can last on its own, but writes need to actively clear the stale entry rather than just waiting it out:

async def update_task(task_id: int, patch: TaskUpdate) -> TaskOut:
    task = await repository.update(task_id, patch)
    await session.commit()                    # 1. persist first
    await redis.delete(f"task:{task_id}")      # 2. only then invalidate
    return TaskOut.model_validate(task)

Invalidating after the commit, not before, matters more than it looks. If you delete (or update) the cache entry before the transaction commits and the transaction later rolls back — a constraint violation, a conflicting concurrent update, a dropped connection — a cache that was pre-emptively updated with the new value would now be actively wrong, holding data that was never actually persisted. A cache that was merely deleted early just repopulates itself from the (unchanged) database on the next read — no harm done, but no benefit either, since you paid the cost of invalidating something that turned out fine. Invalidating strictly after a successful commit means the cache can only ever lag behind the database, never contradict it.

For TTLs, treat the number you pick as "the longest I'm willing to be wrong if invalidation silently fails" — because eventually it will: a deploy that misses an invalidation call, a Redis blip, a code path that updates a row without going through your service layer. A minute or two is reasonable for frequently-read, occasionally-written rows like tasks; data that changes constantly (a live counter) wants a much shorter TTL or no cache at all, while data that rarely changes (a user's plan tier) can tolerate minutes to hours.

One failure mode worth recognizing even though this course won't build the fix for it: the cache stampede. When a very hot key expires, every concurrent request relying on it misses at the same instant, and all of them hit the database simultaneously — for a heavily-read row, that can look like a self-inflicted denial-of-service against your own database. Common mitigations include jittering TTLs slightly per key so hot keys don't all expire on the same tick, having the first request after expiry take a lock and recompute while others briefly wait or serve the stale value ("single-flight"), or probabilistically recomputing a key slightly before its TTL actually runs out. You don't need to implement any of this now — just recognize the symptom (a spike in database load exactly on a cache key's expiry) if you see it later.

3. Query Profiling, Connection Pools & Background Tasks

Caching fixes repeated reads of the same row; it does nothing for a query that's slow every single time. Before reaching for Redis on a slow endpoint, profile it first. Postgres's EXPLAIN ANALYZE run in front of the actual query shows the real execution plan: a Seq Scan on a large table where you expected an Index Scan usually means a missing index on the filtered or joined column, and a query shape that repeats once per row in a loop (an N+1 query, commonly caused by touching a lazily-loaded relationship inside a loop over parent objects) means you need eager loading — selectinload() or joinedload() — not a cache.

Connection pools are the other lever. SQLAlchemy's async engine keeps a pool of live database connections open rather than opening a new one per request:

engine = create_async_engine(
    settings.database_url,
    pool_size=10,      # connections kept open per worker process
    max_overflow=5,    # extra connections allowed under burst load
    pool_timeout=30,    # seconds to wait for a free connection before erroring
)

The number that actually matters is pool_size multiplied by however many worker processes you run in production (Week 13 covers running several with Gunicorn) — that total has to stay comfortably under Postgres's max_connections, or a burst of traffic across workers will exhaust the database's connection slots long before your application code does anything wrong. It's a common cause of "works fine locally, falls over in production" once a service is scaled past a single worker.

Finally, not everything a request triggers needs to finish before the response goes out. FastAPI's BackgroundTasks schedules work — sending a confirmation email, writing an audit log entry — to run after the response has already been sent, without making the caller wait for it:

from fastapi import BackgroundTasks

@router.post("/tasks", status_code=201)
async def create_task(payload: TaskCreate, background_tasks: BackgroundTasks) -> TaskOut:
    task = await repository.create(payload)
    background_tasks.add_task(notify_watchers, task_id=task.id)
    return TaskOut.model_validate(task)

background_tasks.add_task queues notify_watchers to run in the same process right after the response is written to the client. It's a genuine latency win for anything the caller doesn't need to wait on, but it still runs inside your web server's process — a crash before it runs loses it, and a slow background task still consumes that worker's resources while it runs. For work that must survive a crash or needs retries, reach for a real task queue like Celery or RQ instead; BackgroundTasks is the right tool specifically for cheap, best-effort, fire-and-forget work.

4. Hands-on Exercise

5. Knowledge Check

Four quick questions. Expand each to check your answer.