An interactive PyTorch debugger that looks deep inside your neural net
Nansense is an interactive debugger for PyTorch that provides deep visibility into neural network internals, helping developers diagnose issues like vanishing gradients and dead neurons by inspecting activations, gradients, and weights during training.
Background
- Nan-sense is an open-source PyTorch debugger designed to catch and explain "NaN" (Not-a-Number) values during neural network training, which are a common and hard-to-debug failure mode where model weights become invalid numbers.
- It hooks into PyTorch computations to inspect tensors in real time, showing exactly which layer produced the NaN and why, without requiring users to add logging statements manually.
- The tool was announced in early 2025 by Kristjan Kongas and has drawn interest from ML practitioners who have struggled with silent training failures that waste GPU time and obscure root causes.
- Making sense of NaN errors matters because they frequently arise from exploding gradients, division by zero, or log-of-zero operations in deep learning—problems that scale with model size and are notoriously difficult to trace in large, multi-layer networks.