Show HN: Vectordb benchmark – cost (e.g..turbopuffer vs. Zilliz vs. Pinecone
Zilliz has launched VDbbench Leaderboard v2, a cost-aware benchmarking tool comparing vector databases like Turbopuffer, Zilliz, and Pinecone. The methodology and open-source code are available on GitHub, enabling users to evaluate performance relative to cost.
Background
- Vector databases store "embeddings" — numerical representations of text, images, or audio generated by AI models. They power semantic search, RAG (Retrieval-Augmented Generation), and recommendation systems.
- Zilliz is the company behind Milvus, a popular open-source vector DB, and also runs Zilliz Cloud. They host this benchmark, so they are both a participant and the publisher — a point to keep in mind.
- Turbopuffer is a newer, low-cost vector DB provider. Pinecone is a well-established managed vector DB service. Competitors include Qdrant, Weaviate, and Chroma.
- Prior benchmarks focused on speed and accuracy. VectorDBBench v2 adds **cost-aware benchmarking** — measuring price per million queries at a given accuracy level — reflecting growing concerns about runaway vector DB bills in production.