The article provides a formula to calculate GPU memory requirements for running large language models, helping users determine which models fit on their specific GPU hardware. It covers key factors like model parameters, quantization, activations, and context length for the 2026 generation of LLMs.
The article provides a formula to calculate GPU memory requirements for running large language models, helping users determine which models fit on their specific GPU hardware. It covers key factors like model parameters, quantization, activations, and context length for the 2026 generation of LLMs.
The article provides a detailed, first-principles breakdown of occupancy calculation on AMD's MI355X GPU, explaining how to compute thread block occupancy based on hardware limits like shared memory, registers, and workgroup size to optimize kernel performance.
The article provides a detailed, first-principles analysis of occupancy calculations on AMD's MI355X GPU, covering compute unit architecture, wavefront scheduling, and register/shared memory constraints, and offers practical guidance for optimizing kernel occupancy.
The article analyzes the occupancy math and compute throughput for the AMD MI355X GPU, examining how its architectural features like shared memory, register file size, and wavefront scheduling affect kernel occupancy limits and overall performance.
No deep-dive for this story yet — use the button below to generate one.
The article explains how to calculate occupancy for the AMD MI355X GPU from first principles, covering its compute unit architecture, wavefront scheduling, and register/shared memory constraints to guide optimization of kernel occupancy.
The article presents a from-first-principles guide to calculating occupancy on AMD's MI355X GPU, explaining how to derive theoretical occupancy using key hardware limits like wavefront size, shared memory, and register file constraints. It demonstrates the step-by-step math behind occupancy estimation for kernel optimization on AMD CDNA4 architecture.
The article introduces ML-PICO, a practical system for learned image compression, focusing on what design choices matter for real-world deployment rather than just rate-distortion performance. It presents findings on architecture, entropy modeling, and engineering trade-offs to make learned compression viable in practice.
Memory locality—the physical proximity of data in memory—significantly impacts program performance due to CPU caching. Sequential access can be orders of magnitude faster than random access. Strategies like using arrays of structs over structs of arrays improve memory access patterns.
The article provides a formula to calculate GPU memory requirements for running large language models, helping users determine which models fit on their specific GPU hardware. It covers key factors like model parameters, quantization, activations, and context length for the 2026 generation of LLMs.