Week 12: Observability & Production Readiness

1. Spring Boot Actuator

Every service you've built since Week 3 has been a black box from the outside — it either responds or it doesn't. Spring Boot Actuator opens that box: it's a set of production-ready endpoints, built into Spring Boot, that report health, configuration, and runtime metrics without you writing a single controller for them.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Add that one starter and restart, and Actuator is live at /actuator — but by default only /actuator/health is exposed over HTTP. That's a deliberate, safe default, and you'll be turning it up carefully rather than opening everything at once. The endpoints worth knowing:

GET /actuator/health    -> overall app health (UP / DOWN), plus each health indicator
GET /actuator/info      -> static build/app metadata you choose to expose
GET /actuator/metrics   -> index of every metric name Micrometer is recording
GET /actuator/metrics/{name}  -> one metric's current value, e.g. jvm.memory.used
GET /actuator/prometheus      -> all metrics in Prometheus scrape format (Section 3)
GET /actuator/env       -> resolved configuration properties -- DANGEROUS to expose publicly
GET /actuator/loggers    -> view and change log levels at runtime, no redeploy needed

Choosing what to expose

management.endpoints.web.exposure.include controls which endpoints are reachable over HTTP at all. The safe pattern is an explicit allow-list, not * — /actuator/env and /actuator/heapdump can leak database passwords, API keys, and internal hostnames to anyone who can reach the port.

management.endpoints.web.exposure.include=health,info,metrics,prometheus
management.endpoint.health.show-details=when-authorized
management.endpoint.health.show-components=always
management.info.env.enabled=true
info.app.name=@project.artifactId@
info.app.version=@project.version@

Securing actuator endpoints

You already have Spring Security wired up from Weeks 6-7 — reuse it here rather than treating Actuator as a special case. The pattern is the same SecurityFilterChain approach: liveness-style checks stay open for infrastructure to poll, everything else requires authentication.

package com.codeverse.week12.config;

import org.springframework.boot.actuate.autoconfigure.security.servlet.EndpointRequest;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.web.SecurityFilterChain;

@Configuration
public class ActuatorSecurityConfig {

    @Bean
    SecurityFilterChain actuatorSecurityChain(HttpSecurity http) throws Exception {
        http
            .securityMatcher(EndpointRequest.toAnyEndpoint())
            .authorizeHttpRequests(auth -> auth
                .requestMatchers(EndpointRequest.to("health")).permitAll()
                .anyRequest().hasRole("OPS")
            )
            .httpBasic(basic -> {});
        return http.build();
    }
}

2. Custom Health Indicators

Out of the box, /actuator/health aggregates checks Spring Boot can infer automatically — a DataSource being pingable, a Redis connection factory responding. But "is the database reachable" is only half of what your service actually depends on. It also depends on the Notification service from Week 9 being up. A custom HealthIndicator lets you say so explicitly.

package com.codeverse.week12.health;

import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
import org.springframework.web.client.RestClientException;

@Component
public class NotificationServiceHealthIndicator implements HealthIndicator {

    private final RestClient restClient;

    public NotificationServiceHealthIndicator(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://notification-service:8081").build();
    }

    @Override
    public Health health() {
        try {
            restClient.get()
                .uri("/actuator/health")
                .retrieve()
                .toBodilessEntity();
            return Health.up()
                .withDetail("service", "notification-service")
                .build();
        } catch (RestClientException ex) {
            return Health.down()
                .withDetail("service", "notification-service")
                .withDetail("reason", ex.getMessage())
                .build();
        }
    }
}

Spring Boot registers any bean implementing HealthIndicator automatically — no wiring needed. Every indicator's name comes from stripping the HealthIndicator suffix and lowercasing the rest, so the class above shows up as a component named notificationService under /actuator/health.

Checking the cache, too

The database and Redis (Week 11) already get automatic checks from DataSourceHealthIndicator and RedisHealthIndicator, but it's worth confirming what "down" actually means for your app. A Redis outage shouldn't necessarily take the whole service down if you built the cache-aside pattern correctly — reads should fall back to the database. You can reflect that nuance with a custom indicator that reports Status.OUT_OF_SERVICE (a degraded-but-serving state) instead of DOWN:

package com.codeverse.week12.health;

import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.boot.actuate.health.Status;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

@Component
public class CacheHealthIndicator implements HealthIndicator {

    private final StringRedisTemplate redisTemplate;

    public CacheHealthIndicator(StringRedisTemplate redisTemplate) {
        this.redisTemplate = redisTemplate;
    }

    @Override
    public Health health() {
        try {
            redisTemplate.getConnectionFactory().getConnection().ping();
            return Health.up().withDetail("cache", "redis").build();
        } catch (Exception ex) {
            // Redis being down degrades performance (Week 11's cache-aside
            // pattern falls back to the database) but doesn't take the app down.
            return Health.status(Status.OUT_OF_SERVICE)
                .withDetail("cache", "redis")
                .withDetail("reason", ex.getMessage())
                .build();
        }
    }
}

3. Metrics with Micrometer

Health tells you up or down. Metrics tell you how — request latency, error rates, queue depth, how many tasks got completed in the last five minutes. Spring Boot Actuator doesn't collect metrics itself; it delegates to Micrometer, a vendor-neutral metrics facade that plays the same role for metrics that SLF4J plays for logging — your code talks to Micrometer's API, and Micrometer ships the numbers to whichever backend you configure (Prometheus, Datadog, CloudWatch) without your business code knowing which one.

The three core instrument types

Micrometer gives you a small set of instruments, and picking the right one matters:

Counter -> a number that only goes up (or resets to zero)
           e.g. tasks.completed, orders.rejected

Gauge   -> a number that goes up AND down, sampled at read time
           e.g. active.websocket.connections, cache.size

Timer   -> counts AND times occurrences of an event
           e.g. how long POST /tasks took, and how many calls happened

Timing a method with @Timed

For request-level timing, the annotation-driven approach is the least invasive. Enable it once with a TimedAspect bean, then annotate any method:

package com.codeverse.week12.config;

import io.micrometer.core.aop.TimedAspect;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class MetricsConfig {

    @Bean
    TimedAspect timedAspect(MeterRegistry registry) {
        return new TimedAspect(registry);
    }
}

package com.codeverse.week12.service;

import io.micrometer.core.annotation.Timed;
import org.springframework.stereotype.Service;

@Service
public class TaskService {

    @Timed(value = "task.create.duration", description = "Time to create a task")
    public void createTask(String title) {
        // ... persist the task, publish the Kafka event from Week 10
    }
}

A custom business metric: tasks-completed counter

JVM and HTTP metrics come free, but the numbers that matter most to the business — "how many tasks did users actually complete today" — you have to record yourself. Inject the MeterRegistry and increment a counter wherever the domain event happens:

package com.codeverse.week12.service;

import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.stereotype.Service;

@Service
public class TaskCompletionService {

    private final Counter tasksCompletedCounter;

    public TaskCompletionService(MeterRegistry registry) {
        this.tasksCompletedCounter = Counter.builder("tasks.completed")
            .description("Number of tasks marked complete")
            .tag("service", "task-service")
            .register(registry);
    }

    public void markComplete(Long taskId) {
        // ... update task status in the database
        tasksCompletedCounter.increment();
    }
}

Exporting to Prometheus

Add the Prometheus registry dependency and Micrometer starts formatting every metric — JVM, HTTP, and your custom tasks.completed counter — in the text format Prometheus expects, served at /actuator/prometheus (already added to exposure.include in Section 1):

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

curl http://localhost:8080/actuator/prometheus | grep tasks_completed

# tasks_completed_total{service="task-service",} 42.0

4. Structured Logging

Plain-text logs work fine for one service on one machine. They stop working the moment you have two: Task and Notification each write their own System.out-style lines, on different hosts, with no shared way to say "these five lines, across both services, belong to the same request." Grepping two separate log streams by eyeball doesn't scale past a demo.

The fix is structured logging — emitting each log line as a JSON object with consistent field names, instead of a free-form sentence. A log aggregator (ELK, Loki, Datadog) can then index, filter, and join those fields across every service instead of pattern-matching text.

JSON logging with Logback

Spring Boot uses Logback by default. Add the structured-log encoder dependency and swap the console appender's encoder for a JSON one:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>7.4</version>
</dependency>

<configuration>
    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <includeMdcKeyName>traceId</includeMdcKeyName>
            <includeMdcKeyName>spanId</includeMdcKeyName>
            <includeMdcKeyName>requestId</includeMdcKeyName>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="JSON" />
    </root>
</configuration>

Every log line now comes out as one JSON object per line, ready to ship straight into a log aggregator without a fragile regex parser on the other end:

{"@timestamp":"2026-08-02T14:03:11.482Z","level":"INFO",
 "logger_name":"com.codeverse.week12.service.TaskService",
 "message":"Task 482 marked complete",
 "traceId":"6b8f1c2e9a3d4f11","spanId":"a1b2c3d4e5f60718",
 "requestId":"req-9f3e2c1a"}

MDC: attaching context to every log line automatically

Manually passing a request ID into every log statement doesn't scale either. The Mapped Diagnostic Context (MDC) is a thread-local map that Logback reads automatically — set a value once per request, and every log line on that thread includes it without your code repeating itself:

package com.codeverse.week12.web;

import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.slf4j.MDC;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;

import java.io.IOException;
import java.util.UUID;

@Component
public class RequestIdFilter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
                                     FilterChain chain) throws ServletException, IOException {
        String requestId = request.getHeader("X-Request-Id");
        if (requestId == null || requestId.isBlank()) {
            requestId = UUID.randomUUID().toString();
        }
        MDC.put("requestId", requestId);
        response.setHeader("X-Request-Id", requestId);
        try {
            chain.doFilter(request, response);
        } finally {
            MDC.clear(); // never leak context onto the next request on a pooled thread
        }
    }
}

5. Distributed Tracing Basics

The requestId from Section 4 solves logging within one service. It doesn't solve the harder problem: a single user action can trigger a call chain like Task -> Notification (Week 9's synchronous call) or Task -> Kafka -> Notification (Week 10's event-driven path). Following that one logical operation across process boundaries, asynchronously, is exactly what distributed tracing is for.

Trace IDs and span IDs

A trace represents one end-to-end operation — say, "create a task, which triggers a notification." A span represents one unit of work within that trace — the HTTP handler in Task, the Kafka publish, the consumer in Notification. Every span shares the same trace ID, while each span gets its own unique span ID and records which span it's a child of, letting a tracing backend reconstruct the whole call tree.

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-brave</artifactId>
</dependency>
<dependency>
    <groupId>io.zipkin.reporter2</groupId>
    <artifactId>zipkin-reporter-brave</artifactId>
</dependency>

With Micrometer Tracing on the classpath, Spring Boot instruments incoming and outgoing HTTP calls automatically. When Task calls Notification via the RestClient from Week 9, the trace ID and parent span ID are propagated in outgoing request headers (the W3C traceparent header, by default) with no manual plumbing:

management.tracing.sampling.probability=1.0
management.zipkin.tracing.endpoint=http://localhost:9411/api/v2/spans

traceparent: 00-6b8f1c2e9a3d4f11a1b2c3d4e5f60718-a1b2c3d4e5f60718-01
             |  |------ trace id ------------|  |--span id----|

For the asynchronous path — Task publishing to Kafka in Week 10, Notification consuming it later — Micrometer Tracing's Kafka instrumentation injects the trace context into message headers on publish and restores it on consume, so the trace stays intact even though the two spans never share a thread or an HTTP connection.

Closing the loop with structured logs

This is where Section 4 and this section connect: once tracing is on the classpath, Spring Boot automatically populates the MDC with traceId and spanId for every log line — the same MDC keys already wired into the logback-spring.xml encoder above. Search your log aggregator for one trace ID, and you get every log line from Task and Notification that belongs to that single request, in causal order, even though they were written by two separate JVMs on two separate hosts.

6. Hands-on Exercise

7. Knowledge Check

Four quick questions. Expand each to check your answer.