Model Size Scaling in 2023-2031
The article analyzes historical trends in AI model size scaling from 2023 to 2031, projecting that continued exponential growth in model parameters may lead to models with trillions of parameters by the early 2030s, while discussing the limitations and challenges of such scaling, including compute costs and data availability.
Background
- A post on LessWrong (a blog focused on rationality and AI safety). The author forecasts how large AI models will grow and how much computing power will be used to train them between 2023 and 2031.
- Background: For roughly a decade, AI progress followed "scaling laws" — making models bigger and feeding them more data reliably produced better results. By 2023, some researchers questioned whether this was slowing down or hitting fundamental limits. This post examines whether the trend continues.
- Why it matters: Companies are investing billions in AI hardware and training runs based on assumptions about future scaling. If scaling continues, AI capabilities could advance dramatically — and concerns about controlling powerful AI systems become more urgent.
- Key split in the field: Dario Amodei (Anthropic) vs. Ilya Sutskever (SSI Inc.) have taken opposing public positions on whether scaling still works the way it used to.