The AI Superforecasters Are Here
A new generation of AI systems is achieving superhuman accuracy in geopolitical and economic forecasting, outperforming human expert panels. These models combine large language models with structured reasoning techniques and access to real-time data, suggesting AI may soon dominate prediction markets and decision-making support.
Background
- Astral Codex Ten is a popular blog by Scott Alexander (formerly Slate Star Codex), focused on science, rationality, and tech culture. This post reports on a new paper where an LLM was prompted to act as a "superforecaster" — someone who makes unusually accurate predictions about world events.
- "Superforecasters" were identified by psychologist Philip Tetlock in studies showing that a small group of people, using probabilistic thinking and constant updating, consistently beat experts. The term is central to rationalist communities.
- In the experiment, the bot was given real forecasting questions (e.g., "Will North Korea test a nuclear weapon in 2024?") and outperformed both the human superforecaster crowd and a benchmark of aggregated predictions.
- Why it matters: This is one of the strongest early demonstrations that LLMs can compete with top human experts on *predictive reasoning* — a task requiring judgment, uncertainty quantification, and updating on new information. If reliable, it has implications for intelligence analysis, finance, and policy.