What happens when you run a CUDA kernel?
This article explains the detailed process of launching and executing a CUDA GPU kernel, covering steps from CPU invocation and driver interaction to thread block scheduling on streaming multiprocessors (SMs), warp execution, memory access, and synchronization.
Background
- CUDA (Compute Unified Device Architecture) is Nvidia's proprietary platform that lets programmers run code directly on a GPU, not just a CPU. A "kernel" is the Nvidia term for a function written to execute on the GPU.
- To the layperson, a GPU is a graphics card; to a systems programmer, it's a massive parallel processor designed for workloads like AI training, scientific simulation, crypto mining, and game rendering.
- The article walks through the full software/hardware stack: from C++ code → Nvidia's NVCC compiler → PTX (a low-level intermediate instruction set) → SASS (the final machine code that GPU cores actually execute).
- Key concepts covered: thread hierarchy (grids, blocks, warps), memory types (global, shared, local, registers), how the GPU's scheduler hides latency by swapping in ready warps rather than stalling, and how a single line of CUDA code expands into dozens of hardware instructions.