Running 26B and 35B LLMs at Full Speed on €990 of Used Hardware – No Cloud
Two large language models, of 26 billion and 35 billion parameters, can be run locally at full speed using only €990 worth of used hardware, avoiding the need for cloud services and demonstrating cost-effective access to powerful AI inference.
Background
This article is for hobbyists and small teams who want to run large language models (LLMs) — AI models like those powering ChatGPT — on their own hardware instead of renting expensive cloud GPUs. The key trick: using **used enterprise server GPUs** (specifically the NVIDIA Tesla P40, a datacenter card from ~2016 that can be bought secondhand for ~€150–200) and a software stack called **ExLlamaV2**, which is highly optimized for running LLMs efficiently on consumer or older hardware. The "26B and 35B" refer to model sizes (parameters in billions); larger models generally produce better answers but require more memory. By combining two used P40s (total ~€300–400) with a cheap used workstation (€500–600), the author claims you can run these models at speeds usable for chatbots and coding assistants — no cloud subscription or expensive RTX 4090 needed. This is part of a broader trend where falling prices of used enterprise hardware and open‑source LLM software let individuals run AI that previously required corporate budgets.