Natural Language Autoencoders Produce Explanations of LLM Activations

Researchers introduce Natural Language Autoencoders (NLA), a method that converts LLM activations directly into human-readable explanations. Unlike traditional sparse autoencoders that find discrete features, NLAs produce fluent natural language descriptions for any activation, enabling more interpretable analysis of model internals across various architectures and tasks.