How to get GLM 5.2 to 280 tokens per second
Baseten claims to have built the world's fastest API for the GLM-5.2 language model, achieving throughput of 280 tokens per second through optimization techniques including custom CUDA kernels, improved memory management, and efficient model parallelism.
Background
GLM 5.2 is a large language model (LLM) from Zhipu AI, a major Chinese AI company that competes with OpenAI, Anthropic, and Meta. Unlike most Western LLMs, GLM is built in China and uses a different architecture (bilingual, strong on Chinese reasoning). Baseten is a U.S. cloud platform that lets developers deploy open-source models cheaply on their own infrastructure—think of it as a "model hosting service" similar to Replicate or Hugging Face Inference Endpoints. This blog post describes how Baseten engineers tuned GLM 5.2 to run at 280 tokens per second (very fast for a model of this size), which matters because speed and latency determine whether an AI can be used in real-time applications like chatbots, coding assistants, or voice interfaces.