Back Matter · Appendix E
Reading Roadmap & Continued Learning
The canonical papers · Essential tools and codebases · 12-month structured learning path · Community resources for the Principal AI Performance Architect
Book: AI/ML Infrastructure from Silicon to Scale | Author: Venkat Vinjam | Cross-ref: All chapters
How to use this appendix
Papers marked ★ are the ones every principal-level engineer has read and can discuss. Start there. The 12-month learning path in §E.7 maps these papers to specific chapters in sequence. Read each paper after completing the corresponding chapter — theory without system context is abstract; system context without theory is fragile.
E.1 Foundational Papers — GPU Architecture & Memory
| Paper | Year | Why It Matters | Read With |
|---|
| Roofline: An Insightful Visual Performance Model — Williams et al. | 2009 | The roofline model. Every AI infrastructure conversation starts here. ★ | Ch 1 §1.4 |
| Attention Is All You Need — Vaswani et al. | 2017 | The transformer architecture. Mandatory reading. ★ | Ch 2 |
| FlashAttention: Fast and Memory-Efficient Exact Attention — Dao et al. | 2022 | How tiling eliminates the S² memory wall. Changes how you think about GPU kernels. ★ | Ch 4 §4.8 |
| FlashAttention-2 — Dao et al. | 2023 | Work partitioning, reduced non-matmul FLOPs, 2× speedup. ★ | Ch 4 |
| FlashAttention-3 — Shah et al. | 2024 | Hopper-specific: async pipelining, FP8, WGMMA — 75% MFU in practice. | Ch 4, Ch 7 |
| GQA: Training Generalized Multi-Query Transformer Models — Ainslie et al. | 2023 | KV cache reduction with negligible quality loss. Used in every modern LLM. ★ | Ch 2 §2.8 |
| Efficiently Scaling Transformer Inference — Pope et al. (Google) | 2022 | PaLM inference analysis: partitioning strategies, cost models, MQA. ★ | Ch 2, Ch 6 |
E.2 Inference Systems Papers ★
| Paper | Year | Why It Matters | Read With |
|---|
| Orca: A Distributed Serving System for Transformer-Based LLMs — Yu et al. | 2022 | Continuous batching — the most impactful inference optimization (10–30× gain). ★ | Ch 6 §6.2 |
| Efficient Memory Management for LLM Serving with PagedAttention — Kwon et al. | 2023 | PagedAttention, vLLM. Eliminates 87% KV fragmentation. ★ | Ch 6 §6.3, Ch 11 |
| Splitwise: Efficient Generative LLM Inference Using Phase Splitting — Patel et al. | 2023 | First disaggregated prefill-decode paper. The architecture blueprint. ★ | Ch 6 §6.4 |
| Sarathi: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills — Agrawal et al. | 2023 | Chunked prefill — eliminates prefill-decode interference without full disaggregation. | Ch 6 |
| Speculative Decoding: Exploiting Speculative Execution — Leviathan et al. | 2023 | Draft-verify loop, acceptance rate math, 2–4× decode speedup. ★ | Ch 13 §13.1 |
| Medusa: Simple LLM Inference Acceleration Framework — Cai et al. | 2024 | Multi-head speculative decoding — no separate draft model required. | Ch 13 §13.2 |
| RadixAttention: Effective Prefix Caching for LLM Serving — Zheng et al. | 2023 | Tree-structured KV prefix caching — 50–80% TTFT reduction for system prompts. ★ | Ch 11, Ch 6 §6.8 |
| DejaVu: Contextual Sparsity for Efficient LLMs at Inference Time — Liu et al. | 2023 | Dynamic attention and FFN sparsity — skipping 80% of compute at inference. | Ch 6, Ch 8 |
E.3 Distributed Training Papers
| Paper | Year | Why It Matters | Read With |
|---|
| Megatron-LM: Training Multi-Billion Parameter Language Models — Shoeybi et al. | 2019 | Tensor parallelism for transformers. The reference implementation. ★ | Ch 10 §10.3 |
| Efficient Large Scale Language Modeling with Mixtures of Experts — Artetxe et al. | 2021 | MoE scaling laws, training stability. Basis for Mixtral/Grok/Switch. | Ch 13 §13.3 |
| ZeRO: Memory Optimizations Toward Training Trillion Parameter Models — Rajbhandari et al. | 2020 | ZeRO stages 1/2/3 — the memory partitioning foundation for all large training. ★ | Ch 10 §10.4 |
| Reducing Activation Recomputation in Large Transformer Models — Korthikanti et al. | 2022 | Sequence parallelism — removes activation memory from TP, enables longer sequences. | Ch 10 §10.3 |
| GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism — Huang et al. | 2019 | Microbatching for pipeline parallel training — bubble fraction derivation. ★ | Ch 10 §10.2 |
| PipeDream: Fast and Efficient Pipeline Parallel DNN Training — Narayanan et al. | 2019 | 1F1B pipeline schedule — reduces bubble vs GPipe. Used in Megatron-LM. | Ch 10 |
| Llama 2: Open Foundation and Fine-Tuned Chat Models — Touvron et al. | 2023 | GQA at scale, detailed training setup. The reference 70B configuration. ★ | Ch 2, Ch 10 |
E.4 Systems, Kernels & Infrastructure Papers
| Paper | Year | Why It Matters | Read With |
|---|
| Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations — Tillet et al. | 2019 | The Triton compiler — Python-level GPU kernel writing. ★ | Ch 7 |
| CUTLASS: CUDA Templates for Linear Algebra Subroutines — NVIDIA | 2017+ | Production GEMM implementation. Understanding CUTLASS = understanding how CUDA GEMM works. | Ch 7 |
| torch.compile: The Next Generation of PyTorch Compilation — Ansel et al. | 2023 | Dynamo+Inductor+Triton pipeline. How torch.compile actually works. ★ | Ch 9 |
| NCCL: Net Collective Communication Library — NVIDIA | 2016+ | AllReduce algorithms, ring topology, bus bandwidth model. ★ | Ch 14 |
| SmoothQuant: Accurate and Efficient Post-Training Quantization — Xiao et al. | 2022 | Migrate quantization difficulty from activations to weights. Key INT8 technique. ★ | Ch 8 |
| AWQ: Activation-aware Weight Quantization for LLM Compression — Lin et al. | 2023 | INT4 quantization preserving quality. The dominant production INT4 method. ★ | Ch 8 |
| LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale — Dettmers et al. | 2022 | Mixed-precision INT8 inference preserving outlier activations. ★ | Ch 8 |
| GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — Frantar et al. | 2022 | Layer-wise INT4 quantization with Hessian-based calibration. | Ch 8 |
E.5 Profiling, Observability & Production
| Resource | Type | Why It Matters | Read With |
|---|
| Nsight Systems Documentation — NVIDIA | Documentation | The timeline profiler. Learn GPU/CPU/NCCL timeline views. ★ | Ch 17 §17.2 |
| Nsight Compute Documentation — NVIDIA | Documentation | Kernel-level hardware counters, Roofline chart, memory analysis. ★ | Ch 17 §17.3 |
| DCGM User Guide — NVIDIA | Documentation | Fleet-level GPU health monitoring. Field IDs for every metric. ★ | Ch 17 §17.4 |
| Prometheus Documentation + node_exporter — CNCF | Documentation | The metrics collection backbone for every production AI cluster. | Ch 17 §17.6 |
| vLLM Metrics Documentation — vLLM Project | Documentation | vllm:* Prometheus metrics — the inference observability standard. ★ | Ch 6 §6.10, Ch 17 |
| MLPerf Training & Inference Benchmark Results — MLCommons | Benchmark | Verified throughput numbers per GPU. The industry performance reference. ★ | Ch 12 |
E.6 Key Frameworks & Codebases to Study
| Framework | What to Study | Chapter |
|---|
| vLLM (github.com/vllm-project/vllm) | PagedAttention engine, scheduler, quantization integration, Prometheus metrics | Ch 6, Ch 11 |
| TensorRT-LLM (NVIDIA) | Kernel fusion patterns, CUDA Graph integration, FP8 quantization pipeline | Ch 6 §6.5, Ch 9 |
| Megatron-LM (NVIDIA) | 4D parallelism implementation, pipeline scheduling, ZeRO integration | Ch 10 |
| DeepSpeed (Microsoft) | ZeRO stages 1/2/3, gradient checkpointing, inference optimization | Ch 10 §10.4 |
| NCCL (NVIDIA) | Ring AllReduce algorithm, topology detection, environment variable tuning | Ch 14 |
| Triton (OpenAI) | @triton.jit decorator, tl.load/store patterns, autotuning configs | Ch 7 |
| FlashAttention (github.com/Dao-AILab) | CUDA implementation, tiling strategy, forward/backward kernel structure | Ch 4 §4.8 |
| CUTLASS (NVIDIA) | Tile hierarchy (thread block → warp → thread), epilogue fusion patterns | Ch 7 |
| omniperf / rocprof (AMD) | AMD-equivalent of Nsight Compute — kernel profiling on MI300X | Ch 17 |
E.7 The 12-Month Learning Path
MONTHS 1–3: FOUNDATIONS
Month 1: Performance Mindset + GPU Architecture
Read: Ch 1 (Roofline, 3 regimes, MFU)
Ch 3 + Ch 3B (GPU hierarchy, H100→GB300)
Papers: "Roofline" (Williams 2009)
Practice: Download Nsight Compute, run on any kernel, find the roofline chart
Goal: Classify any operation as compute- or bandwidth-bound from specs alone
Month 2: Transformer Architecture as a Performance Problem
Read: Ch 2 (FLOP budget, GQA math, KV cache)
Ch 4 (Memory hierarchy, FlashAttention)
Papers: "Attention Is All You Need", "FlashAttention", "FlashAttention-2"
Practice: Compute LLaMA-3 70B FLOP budget from the model card from scratch
Goal: Derive KV cache size, TPOT minimum, and ridge point before checking answers
Month 3: Inference Architecture
Read: Ch 6 (Inference systems, fleet sizing)
Ch 11 (KV cache engineering)
Papers: "Orca", "PagedAttention" (Kwon 2023), "Splitwise"
Practice: Deploy vLLM locally, examine Prometheus metrics under load
Goal: Explain prefill-decode dichotomy and fleet sizing formula to a colleague
MONTHS 4–6: TECHNICAL DEPTH
Month 4: Kernels + Quantization
Read: Ch 7 (GEMM tiling, Triton)
Ch 8 (FP8, INT4, AWQ)
Papers: "Triton", "SmoothQuant", "AWQ", "LLM.int8()"
Practice: Write a simple Triton GEMM kernel; measure against cuBLAS
Month 5: Training Systems
Read: Ch 10 (4D parallelism, ZeRO)
Ch 14 (Networking, NCCL)
Papers: "Megatron-LM", "ZeRO", "GPipe", "PipeDream"
Practice: Calculate pipeline bubble fraction for PP=4, M=8 vs PP=4, M=32
Month 6: Compilers + Observability
Read: Ch 9 (torch.compile, CUDA Graphs)
Ch 17 (Nsight, DCGM, Prometheus)
Papers: "torch.compile" (Ansel 2023)
Practice: Profile a training step end-to-end with nsys; identify top 3 bottlenecks
MONTHS 7–9: SYSTEMS INTEGRATION
Month 7: Advanced Inference
Read: Ch 13 (Speculative decoding, MoE)
Ch 12 (Benchmarking)
Papers: "Speculative Decoding", "Medusa", "RadixAttention"
Practice: Run MLPerf inference benchmark; compare against published results
Month 8: Infrastructure at Scale
Read: Ch 15 (Storage/IO), Ch 16 (Cluster management), Ch 19 (Resilience)
Practice: Design checkpoint strategy for 1024-GPU training run; calculate overhead
Month 9: Career + System Design
Read: Ch 18 (Interviews, STAR framework)
Practice: Do a 45-minute mock "Design ChatGPT" interview from scratch; time yourself
MONTHS 10–12: MASTERY
Month 10: Deep Review + Gap Filling
Return to any chapter where you could not do all Intermediate review questions
Focus: Any formula you had to look up, look up the derivation until you own it
Month 11: Read the Future
Read: Ch 20 (Future of AI compute), Ch 5 (Thermal/TCO)
Papers: CXL 3.0 spec overview, "Photonic Interconnects for AI" survey
Practice: Model the TCO for a 64-GPU cluster at 3 different cooling configurations
Month 12: Publish and Share
Write a blog post on one technical topic from this book
Open a GitHub issue or PR on vLLM, Triton, or Megatron-LM
Do 3 mock system design interviews with a peer
Goal: Your name should be known in at least one public AI infrastructure forum
E.8 Community & Staying Current
| Resource | What You Get | Signal Quality |
|---|
| arxiv.org/cs.LG + cs.AR | All ML and architecture papers, daily | High — filter by abstract quality |
| Hugging Face Papers (huggingface.co/papers) | Community-curated daily paper highlights | High — crowd-filtered |
| NVIDIA Developer Blog | Technical depth on CUDA, Hopper, NVLink, TRT-LLM | Very high — primary source |
| AMD ROCm Blog | MI300X optimization, ROCm updates | High — primary source |
| MLSys Conference | Top-tier systems + ML papers, annually | Very high — peer reviewed |
| SC / Hot Chips / ISCA | HPC, hardware architecture papers | Very high — peer reviewed |
| GitHub: vllm-project/vllm | Issues, PRs, discussions on production inference | Very high — real problems |
| Tim Dettmers Blog | Quantization, memory efficiency deep dives | Extremely high — practitioner |
| Lilian Weng Blog (lilianweng.github.io) | LLM architecture surveys, RL, alignment | Very high — comprehensive surveys |
| PyTorch Dev Discuss | torch.compile internals, CUDA graph issues | High — core team discussions |
The Principal's Reading Discipline
A principal engineer reads papers differently than a student does. Skim the abstract and conclusion first — if the claim isn't directly relevant to a system you're building or evaluating, move on. When you DO read a paper fully, you should be able to answer: (1) What problem does this solve? (2) What is the key insight? (3) What does this change about how I design or evaluate systems? (4) What are the hidden assumptions? If you can't answer all four, read again.
Appendix E Complete · AI/ML Infrastructure from Silicon to Scale · Venkat Vinjam · 2026