1. A Real Multi-Stage Pipeline
Week 13's pipeline built an image and deployed it in essentially one step. A production-grade pipeline separates concerns into distinct, sequentially-gated stages, so a failure at any point stops the pipeline before it reaches production — the same discipline this course's DevOps counterpart teaches for infrastructure, applied here to an application build.
name: Build, Test, Scan & Deploy
on:
push:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./mvnw test # Week 8's unit + slice tests
- run: ./mvnw verify -Pintegration-tests # Testcontainers integration tests
build-and-scan:
needs: test
runs-on: ubuntu-latest
outputs:
image: ${{ steps.image.outputs.image }}
steps:
- uses: actions/checkout@v4
- run: ./mvnw spring-boot:build-image -Dimage.name=acme/task-service:${{ github.sha }}
- name: Scan image
uses: aquasecurity/trivy-action@0.24.0
with:
image-ref: acme/task-service:${{ github.sha }}
severity: CRITICAL,HIGH
exit-code: "1"
ignore-unfixed: true
- run: docker push acme/task-service:${{ github.sha }}
- id: image
run: echo "image=acme/task-service:${{ github.sha }}" >> "$GITHUB_OUTPUT"
deploy-canary:
needs: build-and-scan
runs-on: ubuntu-latest
environment: production
steps:
- run: ./scripts/deploy-canary.sh ${{ needs.build-and-scan.outputs.image }}
Each job's needs: is what makes this a real gate rather than decoration —
build-and-scan cannot start if test fails, and
deploy-canary cannot start if the Trivy scan fails. This is worth
verifying directly, exactly as Week 13's DevOps counterpart insists: deliberately
break a test and separately reference a known-CVE dependency, and confirm the deploy
job never runs in either case, rather than trusting the YAML at a glance.
Testcontainers integration tests are meaningfully slower than unit and slice tests, since they spin up real containers. Separating them into their own stage (or running them in parallel with a separate job) means a quick unit test failure fails fast, in seconds, instead of every push waiting for the full integration suite before learning about a trivial typo.
2. Blue-Green & Canary Deploys
Week 15's rolling update from the DevOps track replaces Pods gradually, but the new version still eventually receives all traffic with no way to hold it at a fixed, small percentage first. Two stronger patterns for a production Spring Boot service:
Blue-green runs two complete, identical environments — "blue" (currently live) and "green" (the new version) — and switches all traffic from one to the other atomically, typically by repointing a load balancer or Kubernetes Service selector:
apiVersion: v1
kind: Service
metadata:
name: task-service
spec:
selector:
app: task-service
version: green # was "blue" -- this single line switch is the whole cutover
ports:
- port: 80
targetPort: 8080
Both environments run simultaneously before and briefly after the switch, which means
rollback is just as instant as the deploy — flip the selector back to
blue — but it doubles resource usage for the duration, and gives no
opportunity to observe the new version under a small slice of real traffic before it
takes everything at once.
Canary deploys, per Week 16's DevOps service mesh coverage, address exactly that gap by shifting a small, controlled percentage of traffic first:
# Two Deployments, "task-service-stable" (10 replicas) and
# "task-service-canary" (1 replica), both labeled app=task-service
# and selected by the same Service -- Kubernetes load-balances
# across all matching Pods roughly proportional to replica count,
# giving the canary ~9% of traffic with a 10:1 replica ratio.
kubectl scale deployment/task-service-canary --replicas=1
# watch error rate and p99 latency on the canary specifically
# (tag metrics with the "version" label, same idea as Week 16)
kubectl scale deployment/task-service-canary --replicas=5 # ramp up once it looks healthy
This is a cruder mechanism than a service mesh's precise weighted routing — replica count only approximates a traffic percentage — but it needs no additional infrastructure beyond what Week 12–15 already deployed, and it's a legitimate, honest tradeoff for a capstone-scale service to make explicitly, rather than reaching for a full mesh by default.
Blue-green answers "can I switch to the new version instantly, and back instantly, with zero in-between state" — good for a change you're confident in but want an instant abort button for. Canary answers "does the new version actually behave correctly under a small slice of real traffic before I commit to it fully" — good for a change you're genuinely uncertain about. They're not competing options for the same problem.
3. Feature Flags
Every deployment strategy so far ties a code change directly to user-visible behavior changing at the same moment. A feature flag breaks that coupling deliberately: the code for a new feature ships and runs in production behind a conditional, but stays off until explicitly turned on — separating deploy (the code exists and is running) from release (users can actually reach it).
@GetMapping("/api/tasks/{id}/summary")
TaskSummaryResponse getSummary(@PathVariable Long id) {
if (featureFlags.isEnabled("ai-generated-summaries", currentTenant())) {
return aiSummaryService.generate(id); // the new, riskier code path
}
return legacySummaryService.generate(id); // the existing, proven path
}
With a real flagging system (Unleash, LaunchDarkly, or a simple database-backed toggle table for a smaller service), that flag can be enabled for 1% of tenants, or only internal accounts, or only a specific customer who asked for early access — controlled at runtime, with no new deploy required to change who sees it:
@Service
class FeatureFlagService {
private final FeatureFlagRepository flagRepository;
boolean isEnabled(String flagName, Long tenantId) {
return flagRepository.findByName(flagName)
.map(flag -> flag.isGloballyEnabled()
|| flag.enabledTenantIds().contains(tenantId)
|| (flag.rolloutPercentage() > 0
&& tenantId % 100 < flag.rolloutPercentage()))
.orElse(false);
}
}
This is the direct answer to a real limitation in Sections 1–2: a blue-green switch or a canary's replica ratio controls which version of the code serves a request, but both are coarse and infrastructure-level. A feature flag controls which behavior within a single running version a specific user sees, which is a finer-grained, faster, and reversible tool for exactly the case where you want to validate one new feature's behavior, not roll out an entire new build.
Every conditional branch a flag introduces is a maintenance cost and a source of confusing dead code once the feature is either fully rolled out or abandoned. Treat "remove the flag and the losing code path" as a required follow-up task once a rollout is complete — a codebase with dozens of permanently-on flags nobody removed is nearly as hard to reason about as one with no flags at all.
4. Hands-on Exercise
Build a gated pipeline, run a real canary, and gate a feature behind a flag
Apply all three practices to your service, in preparation for the capstone.
Requirements:
- Build the multi-stage pipeline from Section 1 with separate test, build-and-scan, and deploy stages properly gated with
needs:, and verify each gate by deliberately breaking it. - Deploy a stable and a canary version of your service behind a shared Kubernetes Service, ramp the canary's replica count up gradually, and confirm via metrics tagged with a version label that traffic is actually splitting between both.
- Add a simple database-backed feature flag gating one real piece of new behavior, and confirm you can enable it for a specific tenant without deploying new code.
- Simulate a bad canary: deploy a deliberately broken version as the canary, confirm its error rate is visibly worse in your dashboard, and roll it back by scaling it to zero — without touching the stable deployment at all.
Tag every metric and log line with the running version (an environment variable baked into the image at build time is enough) from the very first deploy — retrofitting that tag onto an already-running canary mid-incident is far harder than having it from the start.
5. Knowledge Check
Three quick questions. Expand each to check your answer.
Q1
Why should Testcontainers integration tests run as a separate pipeline stage from unit tests, rather than in the same job?
Why should Testcontainers integration tests run as a separate pipeline stage from unit tests, rather than in the same job?
Integration tests spinning up real containers are significantly slower than in-memory unit and slice tests. Bundling them together means every push, including one with a trivial unit test failure, has to wait for the full slow integration suite before reporting anything — separating them lets a fast unit test failure fail the pipeline in seconds instead of minutes.
Q2
What question does blue-green answer that canary doesn't, and vice versa?
What question does blue-green answer that canary doesn't, and vice versa?
Blue-green answers whether you can switch fully to a new version, and back, instantly with no partial-traffic state — appropriate when you're confident in the change but want an instant rollback option. Canary answers whether a new version actually behaves correctly under real traffic before committing to it fully, by exposing it to only a small slice first — appropriate when you're genuinely uncertain about the change's behavior in production.
Q3
How does a feature flag let you control a rollout more finely than a canary deploy alone?
How does a feature flag let you control a rollout more finely than a canary deploy alone?
A canary controls which version of the running code a request hits, at the infrastructure level — coarse, and typically random with respect to which specific user gets which version. A feature flag controls which behavior within one running version a specific user or tenant sees, decided at runtime with no redeploy, which allows targeting a rollout to a specific tenant, percentage, or cohort with far more precision than replica ratios or traffic weights provide.