Week 18: Distributed Transactions & the Saga Pattern

1. Why Two-Phase Commit Doesn't Fit Microservices

A single database transaction relies on ACID guarantees the database engine enforces directly. The classic distributed equivalent, two-phase commit (2PC), tries to extend that same guarantee across multiple databases: a coordinator asks every participant to prepare to commit, waits for all of them to confirm they can, then tells everyone to actually commit. It works — but it has properties that make it a poor fit for the independent, separately-owned services this course has built since Week 9.

• Blocking — every participant holds its local locks from "prepare" until the coordinator's final decision arrives; if the coordinator or network is slow, every participant's resources stay locked, including rows unrelated services need.
• A single coordinator is a new point of failure — if it crashes between "prepare" and "commit," participants can be left holding a decision they can't resolve on their own.
• Tight coupling — every participating service has to support the same distributed transaction protocol and be simultaneously available for the transaction to complete at all, which works against the entire point of services that can deploy, fail, and scale independently.

The Saga pattern takes a different tradeoff entirely: instead of one atomic operation, a saga is a sequence of local transactions, each in its own service and its own database, where each step publishes an event that triggers the next. There's no cross-service lock held anywhere — the tradeoff is giving up strict consistency in favor of eventual consistency, and taking on the responsibility of undoing already-completed steps if a later one fails.

2. Choreography vs. Orchestration Sagas

Consider placing an order that needs to reserve inventory and charge a payment, spanning three services. Two ways to coordinate the saga:

Choreography — each service reacts to events from the previous step and publishes its own; there's no central coordinator.

// order-service
@Transactional
void createOrder(OrderRequest request) {
    Order order = orderRepository.save(new Order(request, "PENDING"));
    eventPublisher.publish(new OrderCreated(order.id(), request.items()));
}

// inventory-service, listening for OrderCreated
@KafkaListener(topics = "order-events")
void onOrderCreated(OrderCreated event) {
    boolean reserved = inventoryService.tryReserve(event.items());
    if (reserved) {
        eventPublisher.publish(new InventoryReserved(event.orderId()));
    } else {
        eventPublisher.publish(new InventoryReservationFailed(event.orderId()));
    }
}

// order-service, listening for the compensating failure event
@KafkaListener(topics = "inventory-events")
void onInventoryReservationFailed(InventoryReservationFailed event) {
    orderRepository.updateStatus(event.orderId(), "CANCELLED");   // compensating action
}

Orchestration — a dedicated orchestrator service explicitly calls each step in sequence and decides what happens next based on the result, rather than services reacting to each other's events independently:

class OrderSagaOrchestrator {

    void placeOrder(OrderRequest request) {
        Order order = orderService.create(request);
        try {
            inventoryClient.reserve(order.id(), request.items());
            paymentClient.charge(order.id(), request.total());
            orderService.confirm(order.id());
        } catch (InventoryUnavailableException e) {
            orderService.cancel(order.id());                        // compensate
        } catch (PaymentFailedException e) {
            inventoryClient.release(order.id());                     // compensate step 1
            orderService.cancel(order.id());
        }
    }
}

Choreography avoids a central point of coordination and keeps services fully decoupled, but the overall saga's logic ends up scattered across every participant's event handlers — understanding the full flow means reading several services' code. Orchestration makes the entire sequence, including every compensating action, readable in one place, at the cost of the orchestrator needing to know about every step and becoming a coordination dependency of its own. Neither is universally "right" — choreography tends to fit a small number of steps better; orchestration tends to stay more maintainable as the number of steps and failure paths grows.

3. The Transactional Outbox Pattern

Both saga styles above depend on a subtle but critical guarantee: when a service commits a local change, its corresponding event must also get published — reliably, exactly once, even if the service crashes right after committing. Publishing directly inside the same method, right after save(), has a real failure window: if the process crashes between the database commit and the message broker publish succeeding, the local change is durable but the event is lost forever, and every downstream service the saga depends on never finds out.

The transactional outbox pattern closes that gap by writing the event to an outbox table in the same local transaction as the actual business change — both succeed or both roll back together, guaranteed by the single database's own ACID properties, the same guarantee this whole lesson opened by saying doesn't extend across services.

@Transactional
void createOrder(OrderRequest request) {
    Order order = orderRepository.save(new Order(request, "PENDING"));

    OutboxEvent event = new OutboxEvent(
        "Order", order.id().toString(), "OrderCreated",
        toJson(new OrderCreated(order.id(), request.items())));
    outboxRepository.save(event);     // same transaction, same commit-or-rollback
}

A separate poller (or, more robustly, a change-data-capture tool like Debezium reading the database's write-ahead log) then reads unpublished rows from the outbox table and publishes them to the message broker, marking each one published only after a confirmed send:

@Scheduled(fixedDelay = 500)
void publishPendingEvents() {
    List<OutboxEvent> pending = outboxRepository.findByPublishedFalse();
    for (OutboxEvent event : pending) {
        kafkaTemplate.send(event.aggregateType() + "-events", event.payload());
        event.markPublished();
        outboxRepository.save(event);
    }
}

This design accepts at-least-once delivery rather than exactly-once — a crash between sending and marking published can cause a duplicate — which is why every event consumer in a saga needs to be idempotent: processing the same OrderCreated event twice must produce the same end state as processing it once, typically by checking whether that event's ID has already been handled before acting on it.

4. Hands-on Exercise

5. Knowledge Check

Three quick questions. Expand each to check your answer.