Week 14: Async Python & Concurrency

1. The Event Loop, Coroutines & Tasks

Writing async def in front of a function doesn't make it run in parallel, and it doesn't make it run on another thread. It turns the function into a coroutine function: calling it doesn't execute the body at all — it returns a coroutine object, a paused computation waiting to be driven forward. Nothing happens until that coroutine is awaited or handed to the event loop directly.

The event loop is a single-threaded scheduler that keeps a list of in-flight coroutines and switches between them at await points. This is cooperative multitasking: a coroutine only gives up control when it hits an await on something that isn't ready yet (a socket read, a database query, asyncio.sleep). Everything between two await points runs to completion without interruption — there's no OS-level preemption like you'd get with threads, which is exactly why plain Python data structures don't need locks to stay consistent across coroutines the way they would across threads.

import asyncio

async def fetch_user(user_id: int) -> dict:
    print(f"fetch_user({user_id}) starting")
    await asyncio.sleep(0.2)  # stand-in for a network round trip
    print(f"fetch_user({user_id}) done")
    return {"id": user_id, "name": "Ada Lovelace"}

async def main() -> None:
    coro = fetch_user(1)          # just builds a coroutine object -- nothing runs yet
    task = asyncio.create_task(fetch_user(2))  # scheduled on the loop immediately

    print("both created; loop is free to interleave them")
    result_1 = await coro          # now we drive coro forward and wait for it
    result_2 = await task          # task has likely already made progress by now
    print(result_1, result_2)

asyncio.run(main())

asyncio.create_task() is the key move for concurrency: it hands a coroutine to the event loop's scheduler right now, without pausing the current coroutine to wait for it. The returned Task object keeps running concurrently with whatever you do next, and you can await it later to collect its result. Contrast that with await fetch_user(2) directly — that would pause main() until fetch_user(2) finished before moving on to anything else, which is exactly the sequential behavior you want to avoid for independent operations.

2. I/O-Bound vs CPU-Bound Work

asyncio.gather() runs multiple coroutines concurrently and waits for all of them to finish, returning their results in the same order they were passed in. It's the standard way to fan out several independent calls from one handler:

import asyncio

async def dashboard(user_id: int) -> dict:
    profile, tasks = await asyncio.gather(
        users.get(user_id),
        task_repo.list_for_user(user_id),
    )
    return {"profile": profile, "tasks": tasks}

Compare that to the sequential version: profile = await users.get(user_id) followed by tasks = await task_repo.list_for_user(user_id). If each call takes roughly 80ms of real wall-clock time — a network round trip to a database or another service — the sequential version takes roughly 160ms, because the second call can't even start until the first one's response has fully arrived. The gather version takes roughly 80ms, because both calls are in flight on the network at the same time; the event loop starts the second one the instant the first hits its own await and yields control, rather than waiting for it to finish first. This only works because both calls are I/O-bound: the CPU is idle almost the entire time, just waiting for bytes to come back over a socket, and the event loop puts that idle time to use running something else.

Now imagine users.get and task_repo.list_for_user were replaced by two calls to a pure computation — say, hashing a large payload or running a statistical aggregation with no network or disk access at all. Wrapping those in asyncio.gather would buy you nothing. A CPU-bound coroutine has no natural await point to yield at — it's busy the whole time, and Python's Global Interpreter Lock (GIL) only allows one thread to execute Python bytecode at a time regardless of whether that code is written with async/await or not. asyncio gives you concurrency — many things in flight, overlapping their idle time — not parallelism on extra CPU cores. If a coroutine doesn't spend most of its time waiting on something external, async isn't doing anything useful for it, and may even add scheduling overhead for nothing.

A quick self-check that generalizes past this example: if you could run the same call from your terminal with curl and watch it sit there for tens or hundreds of milliseconds before responding, it's very likely I/O-bound and a great candidate for gather. If it would peg a CPU core to 100% for that whole time with no network activity, it's CPU-bound, and you need Topic 3's tools instead.

3. Threads, Processes & Task Queues

Sometimes you're stuck with genuinely blocking code — a legacy database driver with no async variant, a synchronous SDK for a third-party API, or a file-processing library that has no concept of await. Calling it directly inside an async def route blocks the entire event loop for its duration, which means every other concurrent request in that process — not just this one — stalls. loop.run_in_executor() fixes that by handing the blocking call to a worker thread from a ThreadPoolExecutor, freeing the event loop to keep serving other coroutines while that thread waits:

import asyncio

def legacy_blocking_lookup(customer_id: int) -> dict:
    # A synchronous SDK call with no async equivalent -- this really blocks.
    return legacy_sdk.get_customer(customer_id)

async def get_customer(customer_id: int) -> dict:
    loop = asyncio.get_running_loop()
    # None -> use the default ThreadPoolExecutor
    return await loop.run_in_executor(None, legacy_blocking_lookup, customer_id)

Threads share the same GIL, so this doesn't make legacy_blocking_lookup itself run any faster — it still takes just as long. What it buys you is that the event loop's thread is no longer the one sitting idle waiting for it, so other requests keep making progress. For genuinely CPU-bound work, though, a thread pool doesn't help at all, because the GIL still serializes Python bytecode execution across threads. That's what a ProcessPoolExecutor is for: each worker is a separate OS process with its own interpreter and its own GIL, so CPU-heavy functions genuinely run in parallel across cores — at the cost of pickling the arguments and return value across the process boundary, which rules out unpicklable objects like open database connections.

import asyncio
from concurrent.futures import ProcessPoolExecutor

def render_report(rows: list[dict]) -> bytes:
    # CPU-heavy: aggregation + PDF rendering, no network or disk waiting involved
    aggregated = summarize(rows)
    return render_pdf(aggregated)

_report_pool = ProcessPoolExecutor(max_workers=2)

async def generate_report(rows: list[dict]) -> bytes:
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(_report_pool, render_report, rows)

A process pool inside the API keeps the work tied to that API process's lifetime — if the process restarts or redeploys mid-render, the work is gone. Reach for Celery (Week 10) instead once the work is long enough to reasonably outlive a single request-response cycle, needs to survive a deploy, needs retries or scheduling, or should run on separate infrastructure entirely so a spike in report generation can't compete with your API for CPU. The dividing line in practice: a process pool is for CPU-heavy work you're happy to lose and redo if the API restarts; a task queue is for work durable enough that losing it would actually matter.

4. Hands-on Exercise

5. Knowledge Check

Four quick questions. Expand each to check your answer.