#gpu

A first-principles guide to calculating occupancy on the AMD MI355X GPU (CDNA4), covering hardware constraints like compute units, shared memory, registers, and wavefront limits, with examples and the occupancy calculator tool.

Slughorn is a new MIT-licensed, GPU-agnostic font and glyph rendering library designed for use with OpenGL, OSG, Vulkan, and other GPU-driven graphics APIs. It aims to provide high-quality text rendering across multiple graphics backends.

FlashAttention-4 introduces a new algorithm and kernel pipelining design that addresses asymmetric hardware scaling, improving performance on modern GPU architectures by better managing memory and compute resources.

First in a series on optimizing matrix multiplication for NVIDIA's Blackwell GPU. Covers architecture features (GB202/GB203 dies, new SM partitioning, enhanced Tensor Cores) and methodology using CUDA and low-level assembly tuning for peak performance.

This technical report examines latency hiding techniques on Graphics Processing Units (GPUs), analyzing how GPUs hide memory and instruction latency through massive multithreading and warp scheduling. The authors present a detailed study of GPU pipeline behavior and quantify the effectiveness of different latency hiding mechanisms across various workloads.

This article introduces a series on matrix multiplication optimization for NVIDIA's Blackwell architecture, explaining the importance of efficient matrix math for AI workloads and outlining the hardware advancements in Blackwell that enable faster computation compared to previous architectures.

FlashAttention-4 introduces a co-designed algorithm and kernel pipelining approach to improve attention computation on hardware with asymmetric scaling properties, enhancing efficiency and throughput on modern accelerators.

Runway introduces "Borrowing the Night," a system that repurposes idle GPU capacity from inference workloads to run research tasks overnight. By utilizing otherwise wasted compute resources, the approach aims to boost research efficiency without interfering with production inference services.

UATC is a closed-loop controller designed to prevent GPU out-of-memory (OOM) errors by monitoring GPU memory usage and automatically cleaning up unused resources or adjusting allocation.

Shader Benchmark is a tool designed to evaluate the performance of large language models (LLMs) on shader-related tasks, providing a standardized way to test and compare their capabilities in this specific domain.

A crypto miner reflects on a decade of watching Nvidia evolve from a GPU maker for gamers and miners to a trillion-dollar AI powerhouse, tracing the shift from general-purpose CUDA cores to specialized AI hardware and the cultural change from mining mania to the AI boom.

Wgpu v30.0.0 has been released, bringing various updates and improvements to this graphics abstraction layer for Rust. The release includes new features, bug fixes, and changes to the API.

Cerebrium introduces a GPU memory snapshotting technique that reduces cold start times for CUDA workloads from minutes to seconds by saving and restoring GPU state across container restarts, improving efficiency for serverless GPU deployments.

Bargo AI has launched a GPU Compute Tightness Index, a metric designed to measure supply-demand dynamics and pricing pressure in the GPU cloud computing market. The index monitors real-time utilization and availability across major cloud providers to help users assess market tightness for AI workloads.

Nvidia is bringing back older GPU models like the RTX 3060 to address rising RAM demands and help stabilize graphics card prices in the market.

AMD introduces optimized low-latency GEMM kernels for LLM inference on AMD GPUs, reducing prefill and decode latency by up to 3.5x compared to standard implementations. The optimizations target memory-bound operations in transformer-based models, leveraging hardware-specific features to improve throughput and responsiveness in AI workloads.

The article provides an in-depth investigation into the current state of Linux graphics in 2025, examining the various components, drivers, and technologies involved in rendering graphics on the platform.

UATC is a closed-loop controller designed to prevent GPU out-of-memory (OOM) errors during large language model (LLM) training by dynamically managing memory usage.

ZLUDA 6 has been released, enabling users to run unmodified CUDA applications on non-Nvidia GPUs. This update covers work from the first two quarters of 2026 and includes improvements to compatibility and performance for running CUDA software on alternative hardware.

A new feature called TurboPrefill has been introduced to llama.cpp, achieving 2.7× speed improvement over traditional pipeline parallel processing for Llama-3-70B models.

Binning is a process used by manufacturers to sort chips (CPUs, GPUs, RAM) based on quality, performance, and power efficiency after production. Higher-binned parts can achieve higher clock speeds at lower voltages, while lower-binned parts are downclocked or sold as cheaper models. This practice explains why two identical-looking components can perform differently.

The AI industry's GPU shortage is largely artificial, driven by hoarding and inefficient usage rather than genuine scarcity. Smaller, more efficient models and better resource scheduling could alleviate demand. The article predicts a correction where GPU prices fall and the bubble bursts, reshaping AI development priorities.

The article explores how WebGL rendering can be achieved without a physical GPU by using software-based implementations like SwiftShader or Google's Swiftshader, which use CPU to emulate GPU operations. It discusses the performance trade-offs, use cases for server-side rendering, and how tools like Puppeteer can leverage these setups for headless WebGL.

This article explains the detailed process of launching and executing a CUDA GPU kernel, covering steps from CPU invocation and driver interaction to thread block scheduling on streaming multiprocessors (SMs), warp execution, memory access, and synchronization.

The 2026 GPU Price Report from CAST AI analyzes cloud GPU pricing trends, revealing significant cost variations across providers and regions. It highlights that on-demand GPU instances remain expensive, with savings possible through spot instances and reserved capacity, while also noting a shift toward newer, more efficient GPU models impacting overall pricing dynamics.

The article examines the persistent issue of melting power connectors in high-end graphics cards, particularly with Nvidia's RTX 4090, caused by improper cable seating, dust, or high current draw. It explains how these failures occur and offers practical safety tips, such as fully inserting connectors, avoiding sharp bends, and using quality PSU cables to reduce fire risk.

The Reddit post asks about what is known regarding Nvidia's "Feynman" architecture in 2026. The thread likely discusses rumored specs, performance expectations, and potential release timelines for Nvidia's next-generation GPU architecture following Blackwell.

McNUFFT is a Python library implementing the Non-Uniform Fast Fourier Transform (NUFFT) for Apple Silicon GPUs using the MLX framework. It provides both NumPy-like and PyTorch-like APIs, supporting forward and adjoint transforms as well as gradients for machine learning workflows.

A user asks the Hacker News community to compare MacBooks versus dedicated GPUs for running large language models, and how to determine a MacBook's capacity to run a specific model.

The article discusses GPU memory management for AI workloads, explaining how to efficiently allocate and deallocate VRAM (video RAM) to prevent memory fragmentation and leakage. It introduces techniques like proper CUDA caching, tensor garbage collection, and using context managers to ensure memory is freed promptly, helping developers optimize VRAM usage in machine learning applications.