Lecture 3: Networking for AI

tl;dr: In-network aggregation, collective communication algorithm synthesis, and performance simulation for distributed machine learning.

Reading list

1. [NSDI'21] SwitchML: Scaling Distributed Machine Learning with In-Network Aggregation (slides)
2. [NSDI'23] TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches (slides)
3. [NSDI'25] SimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large Language Model Training with Scalability and Precision

All lectures