Skip to content
TopicTracker
From HackerNewsView original
TranslationTranslation

Show HN: Is grep enough? A transparent benchmark for agentic code navigation

The author created a transparent benchmark comparing tree-sitter, grep, and bash tools for agentic code navigation. The benchmark ran 150 isolated tests across 10 large codebases (including Bitcoin, Django, Rails, and Redis) at five complexity levels. All scripts, Docker images, and transcripts are publicly shared.

Background

- The author benchmarks different approaches for letting an AI agent navigate large codebases: **grep** (simple text search), **bash** (shell commands), **LSP** (heavy industry-standard code intelligence), and **tree-sitter** (lightweight parsing library that understands code structure). - They ran 150 isolated tests across 10 major open-source projects (Bitcoin, Django, Rails, Redis, etc.) at 5 difficulty levels. - All scripts, Docker configurations, and transcripts are published for full reproducibility. - Why it matters: as AI coding assistants proliferate, the tools you give them to explore code (simple grep vs. structured parse trees vs. full language server) determine speed, accuracy, and complexity. This is a rare transparent comparison.