GateGPT：FPGA上80MHz下每秒处理5.6万Token的Transformer（KV缓存）
该技术方案展示了在FPGA上以80MHz主频实现Transformer模型推理，通过KV缓存优化达到每秒5.6万Token的处理速度。这一成果证明了低功耗硬件加速器在高效运行大型语言模型方面的潜力，为边缘计算和实时AI应用提供了新的可能性。
相关报道
KV cache compression techniques, including Multi-Query Attention (MQA), Grouped-Query Attention (GQA), Multi-head Latent Attention (MLA), and linear-attention hybrids, have evolved to reduce memory overhead in large language models. These developments have quietly enabled the long context windows required for modern agentic LLM applications by making key-value caching more efficient.