Chip Off the Old Block

- This post is by Scott Alexander (aka "Slate Star Codex"), a prominent psychiatry blogger who writes about tech, science, and rationality on his Substack "Astral Codex Ten." His writing is widely read in Silicon Valley and rationalist communities. - The title "Chip Off the Old Block" is a pun referencing both the idiom (a child resembling a parent) and semiconductor chips — the piece discusses a new AI chip. - The post covers Groq, a startup founded by Jonathan Ross, an original engineer on Google's TPU (Tensor Processing Unit). Groq builds a novel AI chip called the Language Processing Unit (LPU) designed specifically to run large language models like LLama. Unlike Nvidia's GPU-based approach, the LPU uses a simpler, deterministic architecture that prioritizes speed (inference latency) over flexibility, and is not useful for training models. - Groq briefly went viral in early 2024 when its public demo showed LLama running at unprecedented speed (500 tokens/second), but the hype faded as users realized the free tier was limited and the chip doesn't handle training. Scott's post revisits Groq to examine whether it's genuinely promising or just another AI hype cycle.