As AI inference margins shrink, value flows to the hardware supply chain and consumers while model inference becomes a commodity. Frontier AI labs can escape the margin collapse by focusing on managed agents or maintaining a technological lead.
The article argues AI inference is profitable, with estimated costs around $1 per million tokens and API prices far higher, supporting 70-80% gross margins. AI labs charge high margins to subsidize training, but inference itself remains a viable business even if those labs fail.
Pipelines use code-defined control flow for LLM programs; agents let the LLM manage control flow with tools. Pipelines are predictable but rigid, while agents are smarter and more flexible at the cost of unpredictable latency and cost. The author recommends using agents when in doubt.