The future must belong to Small Language Models

- The article argues that the AI industry's obsession with ever-larger Large Language Models (LLMs like GPT-4) is misguided, and that the future belongs to Small Language Models (SLMs) — compact, specialized models that run on local devices instead of massive data centers. - SLMs use techniques like distillation (training a small model to mimic a larger one), quantization (reducing numerical precision), and pruning (cutting unnecessary parameters) to achieve useful performance with far less compute. - Key players pushing SLMs include Microsoft (Phi series), Google (Gemini Nano), Meta (LLaMA 2 small versions), and startups like Mistral AI. Apple's on-device AI features also rely on this approach. - The practical stakes: SLMs can run on phones and laptops without internet, cost far less to operate, use less energy, and offer better privacy by keeping data local. They also democratize AI access for smaller developers and non-English languages. - The "guilt machine" in the title refers to the pressure on AI companies to compete on scale, wasting resources on models that grow exponentially in cost but only marginally in capability.