Show HN: An open source benchmark for prompt-injection detectors
A new open-source benchmark for evaluating prompt-injection detectors has been released. The tool allows developers to test and compare the effectiveness of different detection systems against prompt injection attacks.
Background
- Prompt injection is a security vulnerability where a user crafts input to an AI system (like a chatbot or LLM) to override its intended instructions or safeguards — e.g., making it ignore its "don't reveal private data" rule.<br>- This GitHub repo introduces an open-source benchmark designed to test how well different prompt-injection detectors perform. It provides a standardized dataset and evaluation method so developers can compare tools side-by-side.<br>- Until now, there was no widely agreed-upon public benchmark in this space, making it hard for engineers to know which detectors actually work in practice.<br>- The project is relevant to anyone building or deploying LLM-based applications, where prompt injection remains a critical and unsolved security risk.