Giving AI human-like memory limits (3–7 words) could improve language learning
A new study by researchers at the Max Planck Institute for Psycholinguistics suggests that giving AI language models human-like memory constraints, including the ability to forget, may actually improve their language learning abilities. The researchers argue that forgetting helps filter out irrelevant information, making learning more efficient and robust.
Background
- Modern AI language models (like GPT-4, Llama) are trained on massive text datasets all at once and retain everything; this "batch learning" differs fundamentally from how humans learn language gradually and naturally forget.
- Researchers at the Max Planck Institute for Psycholinguistics in the Netherlands, a leading center for human language processing, ran experiments comparing AI systems with perfect memory against those with artificial "forgetting" mechanisms.
- The study found that AI models with human-like memory constraints—forgetting older information as they learn new things—performed better on language tasks than models that never forgot.
- This challenges the assumption that more data retention is always better, suggesting that forgetting may be a functional feature that improves efficiency and generalization, not a flaw.
- Key implication: building AI more closely modelled on human cognitive processes, including our limitations, could make language AI learn more efficiently from less data—highly relevant as training costs skyrocket.