Week 23: Python Performance — Profiling, the GIL & Free-Threaded Python

1. Profiling with cProfile & py-spy

"This endpoint feels slow" is a hypothesis, not a diagnosis. cProfile, Python's built-in deterministic profiler, instruments every function call and gives you an exact breakdown of where time was actually spent:

terminal — profiling a script directly

python -m cProfile -s cumulative generate_report.py

#          120450 function calls in 2.841 seconds
#    Ordered by: cumulative time
#    ncalls  tottime  cumtime  filename:lineno(function)
#         1    0.002    2.841 generate_report.py:1(<module>)
#         1    0.014    2.612 generate_report.py:12(aggregate)
#      5000    2.401    2.401 generate_report.py:34(compute_score)

tottime (time spent in that function alone, excluding calls it makes to others) is usually the more actionable column — in this example, essentially all 2.841 seconds is spent inside compute_score itself, not in whatever it calls, which tells you precisely where to look first rather than needing to guess across the whole codebase.

cProfile requires instrumenting the process from the start, which isn't always practical for a service already running in production. py-spy solves that by sampling a live process's call stack from outside it, with no code changes and negligible overhead:

terminal — profiling a running FastAPI process without restarting it

py-spy top --pid 4821            # a live, continuously-updating view, like `top`
py-spy dump --pid 4821           # a one-shot stack trace of every thread right now
py-spy record -o profile.svg --pid 4821 --duration 30   # a flame graph over 30s

A flame graph from py-spy record makes the "which function actually dominates wall-clock time" question visual — wide bars are functions where a lot of sampled time landed, and the ability to run it against a real production process under real traffic, without deploying anything different, is exactly what makes it valuable for diagnosing an issue you can't easily reproduce locally.

Profile before optimizing, every time, no exceptions

The function developers guess is slow is very often not the one that actually is — a profiler routinely surfaces the real bottleneck in a place nobody suspected (a logging call formatting a large object every request, a regex compiled fresh on every call instead of once at module load). Optimizing without profiling first risks spending real effort making an already-fast function marginally faster while the actual bottleneck goes untouched.

2. What the GIL Actually Serializes

Week 14 established that the Global Interpreter Lock allows only one thread to execute Python bytecode at a time. The precision in that word "bytecode" matters: the GIL serializes execution of Python-level instructions specifically — it says nothing about what happens while a thread is blocked waiting on something outside the interpreter entirely.

what genuinely runs concurrently despite the GIL

import threading

# CPU-bound pure Python: the GIL serializes this. Two threads running
# this function take roughly the SAME total wall-clock time as one
# thread running it twice, sequentially.
def cpu_heavy():
    total = 0
    for i in range(50_000_000):
        total += i * i

# I/O-bound: releases the GIL while the underlying C code (the socket
# read, the file read) blocks. Two threads running this genuinely
# overlap -- this is threading's real strength even with the GIL.
def io_heavy():
    response = requests.get("https://api.example.com/data")   # GIL released during the wait

The GIL is released for the duration of a blocking system call (a socket read, a file read) and inside many C-extension operations that don't touch Python objects — NumPy's array operations and much of Pandas' internals release the GIL during their actual number-crunching, which is why NumPy-heavy code can genuinely benefit from threads even though "pure Python" CPU-bound code can't. The GIL's real cost is specifically for CPU-bound work written in plain Python — exactly the case Week 14's ProcessPoolExecutor exists to route around, using separate processes with their own independent interpreters and GILs instead.

"Is it CPU-bound?" needs a follow-up: "in Python, or in C?"

A NumPy-heavy aggregation and a pure-Python loop doing the same conceptual work behave completely differently under threading, even though both look "CPU-bound" from the outside. Before reaching for ProcessPoolExecutor to route around the GIL, check whether the actual hot loop is already running inside a C extension that releases the GIL — if it is, threads may already give you real parallelism for free.

3. Free-Threaded Python 3.13

Python 3.13 shipped an experimental free-threaded build (PEP 703) — a build of CPython with the GIL removed entirely, distributed alongside the standard build rather than replacing it. Where Week 14's virtual-threads-style tools worked around the GIL, this removes the actual constraint: genuine multi-core parallelism for pure Python CPU-bound code, with ordinary threading.Thread objects, no process pool or pickling required.

terminal — checking whether you're on a free-threaded build

python3.13t -c "import sys; print(sys._is_gil_enabled())"
# False on a free-threaded build with the GIL actually disabled

This sounds like an unambiguous win for exactly the CPU-bound case Week 14 and Section 2 both flagged as the GIL's real cost — and for genuinely parallelizable pure-Python workloads, it can be. The honest caveats, as of Python 3.13, are just as important:

Ecosystem compatibility is still catching up — many C extensions, including some versions of NumPy, Pandas, and other performance-critical libraries, need to be specifically rebuilt to be safe under free-threading; running an incompatible extension can silently reintroduce race conditions the GIL used to prevent by accident.
Single-threaded performance currently regresses somewhat — the free-threaded build carries real overhead from the more granular locking it needs internally instead of one global lock, so a purely single-threaded workload can run slower on it, not faster.
Removing the GIL exposes real thread-safety bugs — code that "worked" only because the GIL accidentally serialized access to a shared mutable structure can now have genuine race conditions that never showed up before, since nothing was protecting shared state except the GIL's incidental behavior.

Free-threaded Python is a decision to evaluate, not a default to adopt yet

As of Python 3.13 this remains an experimental build, not the default distribution — the right posture for a production service today is watching its maturity closely, not migrating onto it by default. It's a serious answer to a specific, real problem (genuinely CPU-bound, highly parallel pure-Python work), and worth prototyping and benchmarking against for that specific case, but not yet a wholesale replacement for the process-pool pattern Week 14 taught for the same problem.

4. Hands-on Exercise

Hands-on

Profile a real bottleneck, then measure threading with and without the GIL

Find and fix a real hotspot with profiling data, then measure the GIL's actual cost directly.

Requirements:

Profile a genuinely slow endpoint or script from an earlier week's project with cProfile, identify the function with the highest tottime, and fix or optimize it based on that finding, not a guess.
Run your FastAPI service under load and use py-spy record to capture a flame graph while it's handling real requests.
Write a pure-Python CPU-bound function and time it running twice sequentially in one thread versus running twice concurrently across two threading.Thread objects on the standard Python build — confirm the wall-clock time is roughly the same in both cases.
If you have access to a free-threaded Python 3.13 build, repeat the same threaded comparison and note whether the two-thread version now genuinely completes faster than the sequential version.

Hint

py-spy needs elevated permissions to attach to another process on most systems (root on Linux, or specific entitlements on macOS) — check your platform's requirements before assuming a permissions error means the tool is broken.

5. Knowledge Check

Three quick questions. Expand each to check your answer.

What's the key operational advantage of `py-spy` over `cProfile` for diagnosing a production issue?

py-spy attaches to and samples an already-running process from outside it, with no code changes and negligible overhead, so it can profile a live production service under real traffic. cProfile requires instrumenting the process from the start, which means restarting or specifically launching the process under the profiler — often impractical for diagnosing an issue in a service that's already running and that you don't want to interrupt.

Why can NumPy-heavy code benefit from real parallelism across threads even on a standard (GIL-enabled) Python build?

The GIL only serializes execution of Python bytecode; it's released around many C-extension operations that don't touch Python objects directly, and NumPy's array operations release it during their actual number-crunching. That means multiple threads can genuinely run NumPy computation in parallel even under the GIL, unlike an equivalent pure-Python loop, which stays fully serialized because it never leaves the interpreter to release the lock.

Why can removing the GIL on a free-threaded build expose bugs that never appeared on a standard Python build?

Code that mutates shared state across threads with no explicit locking of its own may have only ever worked correctly because the GIL incidentally serialized access to it — the GIL was accidentally acting as a lock nobody wrote. Once the GIL is removed, that accidental protection is gone, and true concurrent access to the same shared mutable state can produce genuine race conditions that a standard build's GIL had been masking all along.