Show HN: role-model, a router for hybrid local/cloud AI
role-model is a routing protocol and runtime for hybrid local/cloud AI that directs requests to the best model based on speed, quality, and cost. It uses task-type metadata from consumer apps to improve routing accuracy and supports local model endpoints.
Background
- A "model router" is software that decides which AI model handles each request — dispatching simple tasks to a cheap/small model and hard ones to a powerful (expensive) model, saving cost and latency.
- Pi (pi.ai) is an AI assistant from Inflection AI. The author built a Pi extension that lets Pi tell the router what a task needs (difficulty, required capabilities), improving routing accuracy.
- "Frontier models" are the most capable models from labs like OpenAI, Anthropic, Google. The author argues routing is most useful when models differ meaningfully (e.g. a tiny local model vs. a cloud model, or a specialized code model vs. a general one).
- Llama-swap, LM Studio, and llama.cpp are tools for running LLMs locally on consumer hardware. The project integrates them to route among local models.
- Rather than a simple "try cheap, escalate to expensive" heuristic, role-model defines a formal protocol: models get roles/domains, requests get task types, enabling more intelligent routing.