Why AI Tokens are so Expensive [video]
The video explains that AI tokens are expensive due to three main factors: the high computational cost of training and running large language models, the vast amounts of data required for training, and the market demand for AI services, which far exceeds current supply, driving up token prices.
Background
- AI models (GPT-4, Claude, Gemini) charge per "token" — roughly 0.75 English words. It's the basic unit they process.
- Generating text is sequential: each new token requires a full pass through a giant neural network (hundreds of billions of parameters) running on expensive GPUs (NVIDIA H100s, ~$30k each). A 500-word answer needs 500 sequential passes.
- This "inference" cost often exceeds training cost because it runs 24/7 serving millions of users.
- The transformer architecture (the "T" in GPT) has O(n²) complexity — double the tokens, quadruple the compute. Long outputs are disproportionately expensive.
- Consumer subscriptions (ChatGPT Plus, Claude Pro) are loss leaders; enterprise API rates reflect the real economics.