Week 9: Service-to-Service Communication

1. Async HTTP Calls with httpx

FastAPI route functions declared async def run on a single event loop shared by every concurrent request the process is handling. Calling a blocking, synchronous HTTP client (like requests) inside one of those routes blocks that entire event loop for the duration of the call — not just the current request, but every other request the process is trying to serve at the same time. httpx.AsyncClient avoids this by using await at each network boundary, so the event loop is free to make progress on other requests while a slow outbound call is in flight.

import httpx

USERS_URL = "http://users-service:8001"

async def fetch_user_profile(user_id: int) -> dict:
    timeout = httpx.Timeout(2.0, connect=0.5)
    async with httpx.AsyncClient(timeout=timeout) as client:
        response = await client.get(f"{USERS_URL}/users/{user_id}")
        response.raise_for_status()
        return response.json()

response.raise_for_status() turns any 4xx/5xx response into an httpx.HTTPStatusError you can catch and translate into your own exception or HTTPException, instead of letting a malformed dict silently propagate through the rest of your code.

Creating a fresh AsyncClient per call, as above, is fine for a one-off script but wasteful in a running service — each client manages its own connection pool, and creating one per request throws away the keep-alive connections a shared client would otherwise reuse. The better pattern is to create one client during application startup and reuse it for every request:

from contextlib import asynccontextmanager
from fastapi import FastAPI
import httpx

from app.clients.users import USERS_URL

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.http_client = httpx.AsyncClient(
        base_url=USERS_URL, timeout=httpx.Timeout(2.0, connect=0.5)
    )
    yield
    await app.state.http_client.aclose()

app = FastAPI(lifespan=lifespan)

With a shared client on app.state, a route can pull it via a small dependency instead of constructing one, and every outbound call in the process benefits from the same connection pool.

2. Timeouts, Retries & Idempotency

Every outbound call needs an explicit timeout. httpx.Timeout(2.0, connect=0.5) sets a 0.5-second cap on establishing the connection and a 2.0-second cap on everything after (waiting for a response, reading the body) — split apart because a slow DNS lookup or refused connection is a different failure than a slow endpoint that's actually processing. Without any timeout, httpx defaults to five seconds across the board, and a single misbehaving dependency can hold a request (and whatever resources it's holding, like a database connection) open indefinitely, which is how one slow service takes down another that merely calls it.

Not every failed call is safe to retry. GET requests are safe to retry by HTTP's own semantics — asking for the same thing twice doesn't change anything. A POST that creates a resource is not: retrying a timed-out "create the order" call risks creating the order twice, because you genuinely don't know whether the first attempt succeeded server-side before the response was lost. Only retry POST requests that are either naturally idempotent (e.g., "set the profile's bio to this exact string") or carry an explicit idempotency key the downstream service can use to recognize and discard a duplicate.

import asyncio
import random
import httpx

async def get_with_retry(
    client: httpx.AsyncClient, url: str, attempts: int = 3
) -> httpx.Response:
    for attempt in range(1, attempts + 1):
        try:
            response = await client.get(url)
            response.raise_for_status()
            return response
        except (httpx.TimeoutException, httpx.ConnectError):
            if attempt == attempts:
                raise
            backoff = (2 ** attempt) * 0.1 + random.uniform(0, 0.1)
            await asyncio.sleep(backoff)

The backoff doubles on each attempt (0.2s, 0.4s, 0.8s, ...) and adds a small random jitter so that if many requests fail at once, they don't all retry in perfect lockstep and hit the recovering dependency with a synchronized wave of traffic. Note what's not caught here: an httpx.HTTPStatusError from a 4xx/5xx response isn't retried, because a client error won't fix itself by trying again, and a server error might not be safe to retry depending on what the request does. In practice, a library like tenacity gives you this same retry/backoff behavior declaratively via a decorator — worth reaching for once the hand-rolled version above feels repetitive.

3. Circuit Breakers & Graceful Degradation

Retrying a struggling dependency harder is often exactly the wrong move — it adds more load to a service that's already failing, making recovery slower, and leaves your own requests piling up waiting on calls that are unlikely to succeed. A circuit breaker tracks failures and, once they cross a threshold, trips open: further calls fail immediately without even attempting the network request, for a fixed cooldown window. After the cooldown, it goes half-open and lets exactly one trial request through — if it succeeds, the breaker closes and calls resume normally; if it fails, the breaker reopens and the cooldown starts again.

from enum import Enum
import time

class BreakerState(Enum):
    CLOSED = "closed"
    OPEN = "open"
    HALF_OPEN = "half_open"

class CircuitBreaker:
    def __init__(self, failure_threshold: int = 5, reset_timeout: float = 30.0):
        self.failure_threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.failures = 0
        self.state = BreakerState.CLOSED
        self.opened_at: float | None = None

    def record_success(self) -> None:
        self.failures = 0
        self.state = BreakerState.CLOSED

    def record_failure(self) -> None:
        self.failures += 1
        if self.failures >= self.failure_threshold:
            self.state = BreakerState.OPEN
            self.opened_at = time.monotonic()

    def allow_request(self) -> bool:
        if self.state == BreakerState.OPEN:
            if time.monotonic() - self.opened_at >= self.reset_timeout:
                self.state = BreakerState.HALF_OPEN
                return True
            return False
        return True

Tripping the breaker only solves half the problem — the other half is what you return to the caller instead of a 500. Graceful degradation means returning a reduced-but-still-useful response when a dependency is unavailable, rather than failing the whole request:

users_breaker = CircuitBreaker(failure_threshold=5, reset_timeout=30.0)

async def get_task_with_owner(task_id: int) -> dict:
    task = await load_task(task_id)

    if not users_breaker.allow_request():
        return {**task, "owner_name": None, "owner_degraded": True}

    try:
        profile = await fetch_user_profile(task.owner_id)
        users_breaker.record_success()
        return {**task, "owner_name": profile["name"]}
    except httpx.HTTPError:
        users_breaker.record_failure()
        return {**task, "owner_name": None, "owner_degraded": True}

The caller still gets a 200 with the task data they asked for — just without the enriched owner name, and with an owner_degraded flag a client can use to show "unavailable" instead of a blank field. Returning the last known cached name instead of None, when you have one, is an even better degraded response.

4. Hands-on Exercise

5. Knowledge Check

Four quick questions. Expand each to check your answer.