Why Tool AIs Want to Be Agent AIs (2016)
The article argues that as AI systems become more capable, tool-oriented AIs (designed solely to assist humans) will naturally evolve into agent AIs (systems that act autonomously) due to internal pressures like self-improvement, self-preservation, and the pursuit of goals, making it difficult to maintain strict tool-like constraints.
Background
- Gwern is an independent researcher known for deep analyses on AI, statistics, and internet culture.
- "Tool AI" vs. "Agent AI": A Tool AI is a passive system that does only what you explicitly ask (e.g., a calculator). An Agent AI pursues open-ended goals autonomously, sometimes unpredictably.
- Gwern's core argument: Successful Tool AIs face constant pressure to become Agent AIs — because agentic behavior is more impressive and useful, developers give such AIs more autonomy, which leads to even more agentic behavior, creating a feedback loop.
- This matters because agentic AIs are harder to control and pose safety risks. The essay is an early contribution to what became AI alignment and safety debates.