那么，下一个词元预测将我们置于何处？
本文探讨了下一代语言模型预训练范式——下一个词元预测（Next-Token Prediction）的现状与未来影响。文章分析了该技术如何在语言建模中带来显著进展，同时也指出了其潜在局限性，包括对长期依赖关系的建模能力不足以及生成文本时的机械性。作者呼吁重新思考当前研究方向，探索更贴近人类认知过程的训练目标。
相关报道
A state-designed worm from 2005 called Fast16 sat undetected on VirusTotal for nearly a decade. It intercepted executable files at the kernel level and silently altered floating-point calculations in high-precision engineering software like LS-DYNA, which was used in Iran's nuclear weapons research. Unlike Stuxnet, Fast16 received little public attention for over twenty years.
Paul Graham reports that Y Combinator startups now have over 75% of their code written by AI, a threshold crossed at least one to two years ago. This parallels a similar transformation at Google, where AI-written code went from 0% to 75% in about two years.
Scientists are increasingly concerned about the potential collapse of the Atlantic Meridional Overturning Circulation (AMOC), a critical ocean current system. Such a collapse could have severe consequences for North America and Europe.
A compromised version of the LiteLLM Python package (version 1.82.8) was briefly available on PyPI, capable of exfiltrating sensitive credentials like SSH keys and cloud secrets. The malicious package affected any project that depended on LiteLLM, though it was only available for about an hour before discovery.
A supply chain attack has compromised the popular npm axios HTTP client library with 300 million weekly downloads. Malicious versions install a remote access trojan, though some users may have avoided infection through version pinning or older installations. Security experts warn this is a live compromise affecting one of npm's most depended-on packages.