Guardian Angels
Gwern discusses "guardian angel" programs — AI agents that watch over users as a safety net against mistakes, security threats, and information overload. The article analyzes the challenges of building such agents, including trust, resource allocation, and balancing autonomy with control, arguing they are increasingly feasible.
Background
- **Gwern Branwen** (Gwern.net) is a renowned independent AI safety researcher who publishes deep, speculative long-form analyses of AI risk, alignment, and transformative AI timelines.
- This article coins and explores the **Guardian Angel** concept: a future AI system built to *protect humanity from other, more dangerous AIs* — essentially a "good" superintelligence tasked with preventing catastrophic outcomes from rogue or misaligned AIs.
- The idea sits within the broader **AI alignment** debate: if advanced AI is developed without robust safety measures, how could a defensive AI (a "guardian angel") reliably identify threats, act preemptively, and avoid itself becoming a threat?
- Gwern treats this as a serious policy and technical proposal, analyzing traps like value lock-in, the difficulty of defining "harm," and the risk that the guardian itself could turn into a totalitarian overseer.