IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures
IatroBench provides pre-registered evidence that AI safety measures can cause iatrogenic harm—unintended negative effects that increase risks instead of reducing them. The study systematically documents such backfires, stressing the need for empirical testing of safety techniques.
Background
- "Iatrogenic harm" means harm caused by a treatment or intervention itself. In this context, it refers to damage or risk introduced by AI safety measures, not by the unconstrained AI.
- This paper, posted on arxiv (a preprint server for research papers) in April 2025, introduces "IatroBench," a benchmark (a standardized test) designed to measure whether common AI safety techniques — such as refusing to answer certain queries, or "red-teaming" (adversarial testing to find vulnerabilities) — end up creating new problems, like leaking private data or degrading performance in critical areas.
- The authors argue that some safety interventions can backfire: for example, a model trained to reject "dangerous" prompts might also reject legitimate medical or legal questions, or the process of safety fine-tuning might inadvertently teach the model to behave less reliably.
- This matters because it challenges the assumption that more safety measures are always better. It raises a trade-off that regulators and developers must face: some safety techniques can do more harm than good.