Co-Failure Ceiling on Mixture-of-Agents Across 67 Frontier Models
This paper introduces the concept of a "co-failure ceiling" in Mixture-of-Agents (MoA) systems, evaluating 67 frontier models to show that combining multiple LLMs can lead to shared failure modes, limiting performance gains beyond a certain threshold.
Background
- This paper tests "Mixture of Agents" (MoA), a technique where multiple AI models work together, with one model acting as a "finisher" that picks the best response from others.
- The authors tested 67 frontier models (top-tier models like GPT-4, Claude, Gemini, Llama, etc.) and found MoA provides minimal gains when used among top models — they call this the "Co-Failure Ceiling," meaning when all models are strong and make similar mistakes, combining them doesn't help.
- Key figures behind the study: researchers from Hugging Face (the leading open-source AI platform) and various universities.
- The finding challenges the assumption that just adding more or better models to an ensemble will keep improving results; it suggests diminishing returns on using multiple frontier models together.