A Multilingual Auditor-Judge Safety Benchmark for Emotional-Support Chatbots
This multilingual benchmark evaluates the safety of emotional-support chatbots by combining an auditor and judge framework. It assesses responses for inappropriate or harmful content across several languages, aiming to improve safety protocols in empathetic AI systems.
Background
- This paper proposes a new safety evaluation framework for chatbots designed to provide emotional support, such as mental-health or crisis-counseling bots.
- The benchmark tests models in multiple languages — a gap in existing safety benchmarks, which are almost entirely English-only.
- It works in two stages: an "auditor" probes the chatbot to try to trigger unsafe behavior (e.g. giving dangerous advice, reinforcing harmful stereotypes), and a "judge" scores whether the response crossed a safety boundary.
- The work responds to a real regulatory and product trend: emotional-support chatbots are being deployed by startups, health systems, and platforms like Character.AI, with high-profile incidents (e.g. a chatbot reportedly encouraging self-harm) fueling calls for stronger safeguards.
- A multilingual focus matters because such chatbots are used globally, but safety alignment (instruction-tuning to refuse harmful requests) is often English-first, leaving non-English users less protected.