Verbalizable Representations Form a Global Workspace in Language Models
A new paper proposes that language models internally organize verbalizable knowledge into a "global workspace"—a shared representation space accessible across different contexts and tasks, analogous to conscious processing in humans. This suggests certain representations are globally broadcast within the model, enabling coherent reasoning.
Background
- This is a research paper from Anthropic's interpretability team (Transformer Circuits), the same group behind prior work on "feature universality" and "dictionary learning" in neural networks. They reverse-engineer the internal computations of large language models like Claude.
- The "Global Workspace Theory" is a famous cognitive science model (Baars, 1988) proposing that conscious thought involves a central "workspace" that integrates information from specialized brain modules. The paper argues LLMs spontaneously develop a computational analog of this.
- "Verbalizable representations" refer to internal model states that directly correspond to what the model can say in words — as opposed to abstract, non-linguistic features. The key claim: these verbalizable features act as a shared bottleneck through which different parts of the model communicate.
- This matters because it suggests a deep convergence between how biological brains and artificial neural networks organize information processing, and offers a mechanistic explanation for how LLMs maintain coherent reasoning across many layers and attention heads.