Week 26 · Capstone: Ship an LLM-Powered Application

1. Project Planning & Scoping

Start exactly the way Week 8's classical ML mini-project did, adapted for an LLM application: write down, before any code, what problem this solves, for whom, and what "working" concretely means.

• What task? — answering questions over a specific document set, drafting a specific kind of content, automating a specific multi-step workflow?
• What data does it need? — a document collection for RAG (Week 22), an instruction dataset for fine-tuning (Weeks 19–20), or neither if prompting (Week 21) alone is sufficient?
• Does it need to take actions, or only generate text? This determines whether Week 23's agent/tool-use machinery is actually necessary, or unneeded complexity.
• What does success look like, measurably? — the Week 24 eval criteria you'll hold the finished system to, decided now rather than invented retroactively to match whatever you happened to build.

Scope deliberately small and real rather than large and hypothetical — a well-scoped assistant over 20 documents you actually evaluate carefully is a stronger capstone than an ambitious, untested idea for "an assistant for everything."

2. Architecture Decisions & Tradeoffs

With the problem scoped, choose your adaptation strategy using Week 20's framework — and be ready to explain the choice, not just make it:

• Prompting alone (Week 21) — if the task is well within a general model's existing capability and just needs the right instructions and examples.
• RAG (Week 22) — if the task depends on specific, possibly-changing knowledge the model wasn't pretrained on.
• Fine-tuning or LoRA (Weeks 19–20) — if the task needs a consistent style, format, or specialized behavior that prompting alone can't reliably achieve.
• Agents (Week 23) — if the task genuinely requires taking actions (looking things up, performing calculations, calling external systems), not just producing text.

Most real capstones combine two or three of these — commonly RAG plus careful prompting, sometimes with a thin agent layer for one or two specific actions. Justify each piece you include: "I added RAG because the task depends on my own document set the base model never saw" is a real architectural decision; "I added an agent because agents are interesting" is not.

3. Full Implementation, from Data to Deployment

Build in the order that lets you validate each layer before adding the next — the same incremental discipline from Week 8's pipeline, now spanning the whole stack:

# 1. Get the core generation working with plain prompting (Week 21) -- cheapest to validate
# 2. Add RAG (Week 22) if your task needs grounding in specific data
# 3. Add fine-tuning/LoRA (Weeks 19-20) ONLY if prompting+RAG demonstrably falls short
# 4. Add agent/tool-use (Week 23) ONLY if the task genuinely requires taking actions
# 5. Wrap the whole thing behind a simple API (Week 25)

This order matters: it's far cheaper to discover that prompting alone was sufficient before spending hours setting up fine-tuning infrastructure you didn't actually need, than to build the most complex version first and simplify afterward.

4. Evaluation, Safety & Cost/Latency Pass

Apply Weeks 24 and 25 in full, not as an afterthought:

• Eval suite — the labeled test set and metrics you defined back in Section 1, run against your finished system, with results reported honestly (including failures).
• Hallucination/groundedness check — if your system uses RAG, verify answers are actually grounded in retrieved content (Week 22, Week 24).
• At least one guardrail — something that blocks or catches a known bad output pattern for your domain (Week 24).
• A red-team pass — 30 minutes deliberately trying to break your own system (Week 24) before calling it done.
• Cost/latency numbers — even rough ones, from Week 25's cost model, so you can speak concretely about what this would cost to run for real.

5. Portfolio Write-Up & Demo

A working system nobody can evaluate from the outside is much less useful as a portfolio piece than a modest system explained clearly. Following Week 8's communication template, adapted for this stack:

• The problem and who it's for — one paragraph, from Section 1's scoping.
• Architecture diagram or description — what pieces (RAG, fine-tuning, agent) you used and, crucially, why (Section 2).
• Evaluation results — your eval suite's scores, honestly reported, including anything it revealed that you fixed or chose to accept as a known limitation.
• Cost/latency at realistic volume — from Section 4's pass, showing you thought about it as a real system, not just a demo.
• A short demo — a recording or a live walkthrough of the system actually working on a real example.

6. The Capstone Project

7. Final Checklist

Before calling the capstone — and the course — done, confirm each of these honestly:

That's the full 26-week path — from a first NumPy array to a deployed, evaluated LLM-powered application. Congratulations on reaching the capstone.