Week 02 — Building LLM Applications¶
Five days of applied sessions turning Week 01 fundamentals into production systems. You build real applications, deploy them with FastAPI, evaluate them with real metrics, and finish with a capstone project ready to demo in an interview.
Prerequisites
Week 02 assumes you completed Week 01 or have equivalent experience: you can call the OpenAI API, you understand what embeddings are, and you've built at least one RAG pipeline.
What You'll Be Able to Do After This Week¶
- Build composable LLM pipelines with LangChain LCEL using async and streaming patterns
- Implement advanced RAG: reranking with CrossEncoder, HyDE, multi-query retrieval
- Fine-tune a model with QLoRA using PEFT and SFTTrainer, then serve it via FastAPI
- Extract structured, validated data from unstructured text using function calling and Pydantic
- Design and implement LangGraph agents with conditional routing, reducers, and checkpointing
- Add observability to LLM systems: LangSmith tracing, cost tracking, latency benchmarking
- Deploy production FastAPI endpoints with SSE streaming, semantic caching, and serverless hosting
- Present a working, evaluated LLM system and explain every architecture decision
Daily Breakdown¶
Day 01 — LangChain + Advanced RAG¶
The most widely used LLM application framework. LangChain solves two problems: composability (chaining LLM calls, tools, and data sources) and portability (switching providers without rewriting logic).
| Topic | What You'll Learn |
|---|---|
| LangChain Overview | Why it exists, core abstractions, LCEL vs legacy chains |
| Chains and Prompts | ChatPromptTemplate, few-shot templates, output parsers |
| Memory | RunnableWithMessageHistory, buffer memory, token-limited memory |
| LCEL | Pipe operator, .ainvoke(), .astream(), RunnablePassthrough, RunnableParallel |
| Practice Exercises | Build a multi-step content pipeline with LCEL |
| Interview Questions | LCEL internals, memory backends, async vs sync |
Basic RAG gets you 70% quality. The remaining 30% comes from addressing specific failure modes: irrelevant chunks, query-document embedding mismatch, and context noise.
| Topic | What You'll Learn |
|---|---|
| Reranking | CrossEncoder scoring, Cohere Rerank API, bi-encoder vs cross-encoder tradeoff |
| HyDE | Hypothetical Document Embeddings — bridge the query-document gap |
| Multi-Query Retrieval | Generate multiple query variants, union results, improve recall |
| Contextual Compression | Strip irrelevant text from retrieved chunks before generation |
| Advanced RAG Patterns | Combining techniques: HyDE + multi-query + rerank in one pipeline |
| Practice Exercises | Add reranking to your Week 01 RAG pipeline, measure Recall@3 improvement |
| Interview Questions | When to use HyDE, reranking costs, failure mode diagnosis |
Day 02 — Fine-Tuning + Function Calling¶
When prompting and RAG aren't enough. Fine-tuning teaches the model a new style, format, or domain — but it's expensive to do wrong. This session gives you a clear decision framework and a working QLoRA pipeline.
| Topic | What You'll Learn |
|---|---|
| Fine-Tuning Overview | Full fine-tuning vs PEFT vs RAG vs prompting — when each applies |
| LoRA and QLoRA | Low-rank decomposition, rank r, alpha, NF4 quantization, memory math |
| PEFT | get_peft_model(), trainable parameter count, merge_and_unload() |
| Training Data Preparation | Chat template format, dataset quality over quantity, deduplication |
| When to Fine-Tune | Decision framework: task type, data availability, latency budget |
| Practice Exercises | Fine-tune a classifier with QLoRA, compare accuracy vs prompt-only |
| Interview Questions | LoRA parameters, training instability, catastrophic forgetting |
Structured output, reliably. Function calling lets you extract validated data structures from unstructured text — no regex, no brittle string parsing.
| Topic | What You'll Learn |
|---|---|
| Function Calling Overview | The 4-step cycle: define → call → execute → return |
| OpenAI Tools API | JSON schema definitions, tool_calls parsing, tool_choice |
| Anthropic Tool Use | tool_use content blocks, tool_result messages, key API differences |
| Structured Extraction | Pydantic models, model_json_schema(), .with_structured_output() |
| Parallel Tool Calls | Multiple tools per response, execution ordering, result assembly |
| Practice Exercises | Extract invoice and support ticket data from raw text |
| Interview Questions | Reliability vs JSON mode, parallel calls, schema design |
Day 03 — AI Agents + LangGraph¶
Chains execute a fixed path. Agents decide the path at runtime. Learn when agents are the right tool and how to build them without the magic — just a loop, a prompt, and tools.
| Topic | What You'll Learn |
|---|---|
| What Are AI Agents | Agents vs chains, the perception-action loop, when complexity is warranted |
| The ReAct Loop | Thought → Action → Observation, stop sequences, loop termination |
| Planning Strategies | Sequential, parallel, tree-of-thought — tradeoffs and failure modes |
| Tool-Calling Agents | Tool registration, execution, error recovery, result formatting |
| Memory Strategies | In-context, sliding window, summarized, vector-backed memory |
| Practice Exercises | Build a research agent with web search + calculator tools |
| Interview Questions | Agent reliability, tool design, failure recovery, token cost |
Stateful, controllable agent graphs. LangGraph gives you explicit state management, conditional routing, and human-in-the-loop — things a raw agent loop can't provide reliably.
| Topic | What You'll Learn |
|---|---|
| LangGraph Overview | Why LangGraph exists, StateGraph vs simple chains, key abstractions |
| Nodes and Edges | Node functions, START/END, add_edge, add_conditional_edges |
| Stateful Graphs | TypedDict state, Annotated reducers, operator.add for accumulation |
| Conditional Routing | Router functions, quality gates, retry loops, cycle detection |
| Multi-Agent Orchestration | Supervisor patterns, parallel subgraphs, MemorySaver, interrupt_before |
| Practice Exercises | Build a writer-critic loop that reruns until quality score ≥ 0.80 |
| Interview Questions | State reducers, checkpointing, human-in-the-loop design |
Day 04 — LLMOps + Deployment¶
You can't improve what you can't measure. Add observability to your LLM systems — tracing every request, tracking every dollar, catching regressions before users do.
| Topic | What You'll Learn |
|---|---|
| Tracing and Logging | Structured logging, span tracing, what to capture per request |
| LangSmith | @traceable, dataset creation, evaluate(), regression detection |
| Cost Tracking | Token counting with tiktoken, pricing dict, per-request cost logging |
| Latency Optimization | Parallel calls, exact-match cache, model selection, streaming perception |
| Observability | Metrics to track, alerting thresholds, dashboard design |
| Practice Exercises | Instrument your RAG pipeline with LangSmith and cost tracking |
| Interview Questions | Monitoring strategy, cost control, regression detection |
A model that isn't deployed isn't a product. Wrap your pipelines in production FastAPI services with streaming, async, caching, and serverless hosting.
| Topic | What You'll Learn |
|---|---|
| FastAPI Wrappers | lifespan, Pydantic request/response models, APIKeyHeader auth |
| Streaming Responses | StreamingResponse, SSE format (data: {}\n\n), X-Accel-Buffering: no |
| Async Patterns | AsyncOpenAI, asyncio.Semaphore, asyncio.gather, BackgroundTasks |
| Serverless Deployment | Modal (@app.function), AWS Lambda + Mangum, Fly.io cold start tuning |
| Caching Strategies | SHA256 exact-match cache, semantic cache with cosine similarity threshold |
| Practice Exercises | Add streaming + semantic cache to your RAG endpoint |
| Interview Questions | SSE vs WebSocket, async client design, cache invalidation |
Day 05 — Capstone + Mock Interview¶
Everything comes together. Design, build, evaluate, and present a production LLM application integrating at least four course components.
| Resource | Purpose |
|---|---|
| Project Brief | Three project options with full scope definitions |
| Architecture Design | Design doc template, Mermaid diagrams, component selection guide |
| Implementation | Complete working code for all three options |
| Evaluation and Testing | Eval scripts per project option, API endpoint testing |
| Presentation Guide | 5-minute demo structure, what interviewers listen for |
| Submission Checklist | Self-assessment rubric, required vs stretch deliverables |
Three project options:
- Option A — Customer Support Bot with RAG — ChromaDB retrieval, streaming FastAPI, RAGAS evaluation
- Option B — Research Agent with LangGraph — planner + researcher + writer + critic with quality gating
- Option C — Document Intelligence Pipeline — multi-format ingestion, structured extraction, LLM-as-judge
Convert two weeks of learning into something usable in a job search. Resume polish, GitHub portfolio, rehearsed answers, and a full mock interview with feedback.
| Resource | Purpose |
|---|---|
| Resume Checklist | Skills section format, bullet formula, common mistakes |
| Portfolio and GitHub | Repo structure, README template, what interviewers actually look at |
| Technical Interview Questions | 12 Q&As across RAG, APIs, agents, deployment, fine-tuning, evaluation |
| System Design Practice | 3 full problems with clarifying questions, reference architectures, tradeoffs |
| Mock Interview Script | Complete 45-minute mock with 10 questions, interviewer notes, debrief template |
Week 02 Tech Stack¶
pip install langchain langchain-openai langchain-anthropic langchain-community \
langchain-cohere langgraph langsmith \
transformers peft trl bitsandbytes accelerate \
fastapi uvicorn sse-starlette httpx \
ragas sentence-transformers chromadb \
modal mangum tiktoken
GPU required for fine-tuning
Day 02 Part 1 fine-tuning exercises require a CUDA GPU with at least 16 GB VRAM for QLoRA on a 7B model. Use Google Colab (A100 runtime) or Kaggle (T4 GPU) if you don't have local hardware.
Week 02 Assignment¶
Build one of three capstone options to production quality:
- Working FastAPI endpoint with streaming
- Async patterns used correctly throughout
- At least 4 course components integrated
- Quantitative evaluation with ≥ 10 test cases and results in README
Full brief: Week 02 Assignment
What this week proves
A working, evaluated, documented project that you built from scratch is worth more in an interview than any certificate. When you can say "faithfulness score 0.87 on 50 test cases, p95 latency 1.2s, cost $0.003 per request" — that's the conversation that gets offers.
Previous: Week 01 — LLM Fundamentals