We Don't Understand Neural Networks at the Algorithmic Level
Neural networks remain poorly understood at the algorithmic level, despite progress at the implementational and computational levels. The author argues that current explanations lack a clear algorithmic understanding of how neural networks actually compute, highlighting a gap in neuroscience and AI research.
Background
- Alon Zakai ("kripken") is the creator of Emscripten (compiles C/C++ to WebAssembly) — a well-known compiler engineer writing as an informed outsider to neuroscience.
- The post uses David Marr's classic three-level framework (computational → algorithmic → implementational) to argue that we understand neural networks at the lowest (synapses/neurons) and highest (behavior) levels, but we lack the middle algorithmic level — what computations a trained network actually performs internally, akin to having a compiled binary with no source code.
- Without algorithmic understanding, we cannot predict failure modes, transfer learning between networks, or prove safety properties. The author draws a parallel to early computing: programmers once worked only in machine code before higher-level algorithmic abstractions (sorting, hashing, etc.) emerged.