Tripping on Acid

- "Acid" here refers to a hardware accelerator called Groq (not the chatbot Grok), whose chips are designed to run large language models (LLMs) at extremely high speeds. - The post is a technical comparison between running the same LLM prompt on a Groq-powered system ("Acid") vs. a conventional GPU setup. The author measures not just speed but "time to first token" and "tokens per second" — metrics that determine whether an AI feels instantaneous or sluggish in conversation. - Groq's key innovation is a custom chip architecture (the LPU) that avoids the memory bottlenecks of GPUs, enabling much lower latency. This matters because for interactive use—like voice assistants, coding copilots, or real-time agents—a model that answers in milliseconds feels qualitatively different from one that takes seconds. - The post is dated 2026, placing it in a near-future speculative or fictional timeline, exploring how hardware advances might reshape the experience of using AI.