Modern GPU Programming for MLSys
This course covers modern GPU programming techniques for machine learning systems, including CUDA, Triton, and other GPU programming models, aimed at helping developers build and optimize efficient ML workloads on GPUs.
Background
- This is a technical course/guide from MLC.ai (Machine Learning Compilation), a group focused on bridging ML and systems programming.
- Covers how to write high-performance GPU kernels using approaches beyond basic CUDA: shared memory, tensor cores, warp-level primitives, and compiler-based optimization (TVM, Triton).
- Matters because modern ML models (LLMs, diffusion models) are bottlenecked by GPU kernel efficiency, not just model architecture. Understanding these techniques is key for roles in ML infrastructure, inference serving, and AI hardware.
- Prior context: Most ML practitioners use PyTorch/TensorFlow and never write custom GPU code. This material targets people who want to optimize or deploy models on GPUs, competing with libraries like vLLM, TensorRT, or FlashAttention.