Understand and design AI systems by focusing on constraints rather than models
This open textbook series covers the fundamental principles of machine learning systems, scaling methods, autonomous agents, and physical AI. It examines whether models with high benchmark scores can actually fit in memory, meet latency requirements, and operate safely within power limits. Readers calculate how hardware constraints determine architecture, write code directly, and model large-scale clusters using simulators.
Constraints determine architecture
The core idea of this textbook is that constraints determine architecture. Whether on a microcontroller with 256KB of memory or a data center accelerator, memory bandwidth, network latency, energy, and failure rates govern system design. The author realized this while researching TinyML at Google and applies it consistently across all volumes. Based on physical laws that do not change when frameworks or chips change, readers can develop system thinking that is not tied to specific tools. This is also an attempt to establish 'AI engineering' as a new academic field alongside software engineering and computer engineering.
Expanding system boundaries across four volumes
The textbook is structured into four volumes based on the perspective that the cost of failure increases as the scope of the system expands. Volume 1 deals with how models run efficiently on a single machine and is awaiting print publication from MIT Press. Volume 2 covers distributed clusters with thousands of connected accelerators and data center scaling. Volume 3 addresses the governance of agent systems that act autonomously, while Volume 4 covers Physical AI that interacts with the physical world. Volumes 3 and 4 are currently in draft stage, and since content changes frequently, it is better to approach them for providing feedback rather than for citation or course materials. Each volume continues the questions left by the previous one, expanding the system boundary from a single machine to physical action.
Reading is not enough: a set of practical tools
This repository provides not only a textbook but also practical tools that concretize the concepts.
- TinyTorch A 20-step modular project for building a machine learning framework from scratch.
- MLSys·im A tool that simulates memory and energy limits of large-scale clusters to support the textbook's calculations.
- Hardware Kits Kits that allow users to experience physical constraints by deploying models to actual embedded devices.
- StaffML Provides system design interview questions and practice problems for cloud, edge, mobile, and TinyML domains.
A learning path verified in practice
This material started as course notes for Harvard CS249r in 2023 and has evolved through rewriting each semester. It is adopted and used by several universities, including UC Berkeley, Columbia, and Georgia Tech, and has been publicly revised through more than 16,000 commits. All figures are calculated with Python code, and a test suite based on MLSys·im verifies unit consistency and citations. Engineers from the industry review long-form content to ensure it aligns with real systems, reducing the gap between theory and practice. Readers can start learning with only Python programming and basic machine learning concepts, without background knowledge in computer architecture or distributed systems.
Things to check before use
This repository is an educational open-source project. While Volumes 1 and 2 are stable, Volumes 3 and 4 are in draft status, so you must check the latest status before citing them in courses or official documents. Licenses vary by component: the textbook is CC BY-NC-ND 4.0, TinyTorch is MIT, and StaffML is AGPL v3, among others. Therefore, you must check the LICENSE file in each directory for commercial use or modification. In particular, StaffML is subject to a strong Copyleft license that requires publishing modifications to distributed services, so license conditions should be carefully reviewed when adopting it in a corporate environment. Additionally, since all figures are based on specific hardware specifications, it is recommended to reproduce calculations directly using MLSys·im when applying them to your own system.
