Foundation Model Engineering: From Theory to Production

Foundation Model Engineering is a technical textbook for readers who want to understand how modern foundation models actually work, why the stack evolved the way it did, and what engineering trade-offs appear when those ideas meet real systems. This project is written primarily for AI engineers and research-oriented readers who want to move past surface-level API usage and build a deeper mental model of architectures, training pipelines, inference systems, retrieval stacks, evaluation loops, and agentic workflows. The goal is not to provide scattered tips or isolated definitions. The goal is to explain the historical flow, mathematical ideas, and systems constraints that connect topics like attention, MoE, RLHF, multimodality, long-context serving, RAG, and agents into one engineering narrative. Why read this If you have ever wondered why the field moved from RNNs to Transformers, why some models are dense while others are sparse, why inference systems care so much about KV cache and batching, or why evaluation and alignment are product problems rather than just research topics, this book is meant to help you connect those dots. Instead of treating each topic as an isolated trend, the book tries to show how modeling ideas, systems constraints, and product requirements shape one another. The payoff is not just more terminology. It is better engineering judgment. Who this is for AI Engineers Readers building or evaluating LLM systems, inference stacks, RAG systems, or agentic products. Research-Oriented Readers Readers who want a broad but technically grounded understanding of the foundation model landscape, including current architecture and systems trends. What to expect You will find rigorous conceptual explanations, concept-focused PyTorch examples, short quizzes for consolidation, and interactive visualizers for topics that are easier to understand by manipulating them directly. The material is designed to help you reason about quality, memory, throughput, latency, scaling, and alignment trade-offs, not just memorize terminology. This is not a lightweight beginner introduction. If you are looking for a first overview of AI or a prompt-engineering-only guide, this book will probably feel denser than necessary. It is intentionally written for readers who want depth. A Living Document AI changes extremely quickly, so some details in a project like this may need revision as new papers, systems, and products appear. If you spot an outdated section, an awkward explanation, a typo, or a better reference, contributions are always welcome. Pull requests that improve accuracy, pedagogy, examples, localization, or overall clarity are appreciated. The goal is for this to remain a useful long-term resource, not a frozen snapshot. Table of Contents 1. The Evolution of Intelligence 1.1 Symbolism vs. Connectionism 1.2 The Power of Representation 1.3 Deep Learning Paradigms 1.4 The Bitter Lesson 2. The Sequence Modeling Era 2.1 Markov Chains to RNNs 2.2 Vanishing/Exploding Gradients 2.3 The Dawn of Attention 2.4 CNNs for NLP 3. The Transformer Deep Dive 3.1 Self-Attention Mathematics 3.2 Multi-head Attention (MHA) 3.3 Position Encoding Strategy 3.4 Layer Normalization & Residuals 3.5 Complexity Analysis 4. LLM Architectures & Paradigms 4.1 Encoder-only (BERT-style) 4.2 Decoder-only (GPT-style) 4.3 Encoder-Decoder (T5/BART) 4.4 Hybrid & Prefix LM 4.5 Various LLM Architectures & Latest Trends 5. Scaling Mixture of Experts (MoE) 5.1 Sparse vs Dense Models 5.2 Routing Algorithms 5.3 Expert Parallelism 5.4 Collapsing & Load Balancing 5.5 Case Study 6. Foundation Model Pre-training 6.1 Data Engineering at Scale 6.2 Tokenization Science 6.3 Large-scale Training Stability 6.4 Infrastructure 6.5 Synthetic Data for Pre-training 7. Training Optimization & Systems 7.1 Data Parallelism (DP/DDP) 7.2 ZeRO (Zero Redundancy Optimizer) 7.3 Model & Pipeline Parallelism 7.4 Flash Attention 1, 2, 3 8. Scaling Laws & Compute Optimality 8.1 The Power Law 8.2 Chinchilla Optimality 8.3 Over-training vs Optimal-training 8.4 Transfer Learning & Generalization 8.5 Continued Pre-training & Domain Adaptation 9. Post-training: SFT & Instruction Tuning 9.1 SFT Fundamentals 9.2 Dataset Quality vs Quantity 9.3 Parameter-Efficient Fine-Tuning (PEFT) 9.4 Prompt Engineering as SFT 9.5 Synthetic Instructions & Self-Instruct 10. Alignment: RLHF & Direct Preference 10.1 Human Feedback Loop 10.2 PPO (Proximal Policy Optimization) 10.3 DPO (Direct Preference Optimization) 10.4 KTO & IPO 10.5 Alignment Tax 11. Multimodal Learning 11.1 Vision-Language Bridges 11.2 Audio & Speech Integration 11.3 Unified Multimodal (Any-to-Any) 11.4 Image Diffusion Models 11.5 Video Generation Foundations 11.6 Commercial Video Models & Future Directions 12. LLM Inference Optimization 12.1 KV Cache Management 12.2 PagedAttention (vLLM) 12.3 Continuous Batching 12.4 Speculative Decoding 12.5 Long Context Serving 12.6 Serving Policies, SLOs, and Fallbacks 13. Model Compression & Quantization 13.1 PTQ vs QAT 13.2 Quantization Methods 13.3 Weight Sparsification 13.4 Knowledge Distillation 13.5 Advanced Quantization 14. RAG (Retrieval Augmented Generation) 14.1 From Lexical to Semantic Search 14.2 Vector Indexing & DB Solutions 14.3 Advanced Retrieval & Reranking 14.4 RAG Orchestration 14.5 GraphRAG & Ontology 14.6 RAG Failure Modes and Operational Design 15. Reasoning & Search-time Scaling 15.1 Chain of Thought (CoT) 15.2 Tree/Graph of Thoughts 15.3 Search-time Compute 15.4 Verifiers & Reward Models 16. Agentic AI & Tools 16.1 Function Calling & Tool Use 16.2 Autonomous Agents 16.3 Self-Improving Agents 16.4 Multi-agent Collaboration 16.5 Long-term Memory for Agents 16.6 Agent Reliability, Recovery, and Guardrails 17. AI Evaluation & Benchmarking 17.1 Academic Benchmarks 17.2 LLM-as-a-Judge 17.3 Elo Rating & Leaderboards 17.4 Contamination Issues 17.5 Production Evaluation and Release Gates 17.6 Commercial Model Benchmarks 18. AI Safety & Alignment Research 18.1 Red Teaming 18.2 Jailbreaking & Defense 18.3 Hallucination Detection 18.4 Scalable Oversight 19. Interpretability & Science of LLMs 19.1 Mechanistic Interpretability 19.2 Logit Lens & Attention Visualization 19.3 Probing Classifiers 19.4 Sparse Autoencoders (SAE) 20. Next Generation: SSM & Beyond 20.1 State Space Models (SSM) 20.2 Mamba & S6 20.3 Linear Attention 20.4 Neural Networks as Programs 20.5 Multi-Token Prediction 20.6 Diffusion-based LLMs 20.7 Path to AGI & World Models