Deriving SVD without even aiming at it
The article presents an intuitive derivation of the singular value decomposition (SVD) by exploring geometric connections between linear transformations, showing how the SVD emerges naturally from understanding how a matrix acts on the unit circle without directly aiming to decompose the matrix.
Background
- Singular Value Decomposition (SVD) is a fundamental matrix factorization technique in linear algebra, widely used in machine learning, data compression, signal processing, and recommendation systems (e.g., Netflix's algorithm or Google's PageRank).
- This post offers an intuitive, visual derivation of SVD — how any matrix can be decomposed into three simpler matrices (U, Σ, V^T) that capture rotation, scaling, and another rotation — without starting from the standard textbook definition.
- The author, Alex P., runs "Still Thinking," a math-and-tech blog that explains advanced concepts from first principles, often with interactive visualizations.
- For readers unfamiliar with linear algebra: a matrix is a rectangular array of numbers; "decomposition" means breaking it into parts that are easier to analyze. SVD is especially powerful because it works on any matrix (not just square ones) and reveals the "hidden structure" of the data it represents.