New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is bu...

Andrew Ng announced a new short course built with Red Hat on serving LLMs efficiently. The course covers quantization to reduce memory footprint, serving models with vLLM for concurrent requests, and benchmarking to balance speed, cost, and accuracy.