TurboPrefill: 2.7× faster than llama.cpp Pipeline Parallel on Llama-3-70B

- llama.cpp is a popular open-source C/C++ project that lets you run large language models (LLMs) like Meta's Llama on consumer hardware, not just expensive datacenter GPUs. - Pipeline Parallelism (PP) is a standard technique for running a single big model across multiple GPUs: each GPU handles a different layer, passing results down the chain like an assembly line. - This PR (pull request) proposes TurboPrefill, a new technique that speeds up the "prefill" phase — where the model first reads your prompt — by 2.7× on a 70-billion-parameter Llama model compared to the existing pipeline-parallel approach in llama.cpp. - The prefill phase is often a bottleneck because it processes the whole prompt at once, while the subsequent "decoding" phase generates one token at a time. Making prefill faster means lower latency before you see the first word of a response. - The author claims the improvement comes from more efficiently keeping all GPUs busy during prefill, rather than having some sit idle waiting for others in the pipeline.