Ask HN: Best AI Gateway?
A Hacker News user asks for recommendations on the best AI gateway, noting they currently use Anthropic's router but are considering OpenRouter, Vercel AI Gateway, or Cloudflare's offering. They express concerns about OpenRouter's 5.5% platform fee and lack of prompt caching support, while noting that many "best gateway" articles are written by companies that sell their own gateways.
Background
- An **AI gateway** is middleware between your app and LLM providers (OpenAI, Anthropic, etc.). It handles routing, failover, caching, API key management, and cost/usage tracking.
- The user (on Hacker News) currently calls Anthropic's API directly from **Val Town** (a platform for running serverless scripts). They're evaluating alternatives to handle multiple models and customers.
- **OpenRouter**: popular gateway with many models, but charges a 5.5% fee and lacks prompt caching (a feature that saves money by reusing common text prefixes across requests).
- **Vercel AI Gateway** and **Cloudflare AI Gateway**: both from major cloud platforms. The user is a Cloudflare customer but skeptical of reviews from companies that sell their own gateway.
- "New VS Code forks" = a joke that just as every company forked VS Code, now every company builds an AI gateway.