Show HN: Evaluation Context Protocol (ECP)
Evaluation Context Protocol (ECP) is a vendor-neutral, open protocol designed for portable evaluations of AI agents. It enables testing of agent outputs, tool calls, and audit context across different frameworks, models, evaluation platforms, and CI systems.
Background
- **Evaluation Context Protocol (ECP)** is a proposed open standard for testing AI agent outputs and behavior across different frameworks, models, and platforms — similar to what LSP did for code editors.
- The problem: today, each AI framework (LangChain, CrewAI) and evaluation platform has its own ad-hoc way to log tool calls and internal agent decisions, making it hard to reuse tests or switch tools.
- "Evaluator-visible audit context" means evaluators can see the agent's internal actions — which tools it called, what it returned — not just its final answer.
- It's vendor-neutral (no ties to a specific LLM provider), following the pattern of Anthropic's Model Context Protocol (MCP) which standardized how AI models connect to external data.
- Why it matters: reliable, reproducible evaluation is essential for production AI agents. A common protocol means you don't have to rewrite evaluation logic every time you switch frameworks or evaluation tools.