M* (M-Star): A Modular, Extensible, Serving System for Multimodal Models
M* (M-Star) is a modular, extensible serving system developed at Stanford for efficiently deploying multimodal AI models. It is designed to support a wide range of model architectures and hardware backends, enabling flexible and high-performance inference for research and production use.
Background
M* (pronounced "M-Star") is an open-source serving system developed at Stanford University for running large multimodal AI models (models that process images, text, video, etc.) in production. It is designed to be modular and extensible, meaning developers can swap in different model architectures or hardware backends without rewriting the entire system. This matters because deploying multimodal models efficiently is far more complex than serving text-only models — M* aims to make that easier for researchers and engineers, similar to how vLLM or TensorRT-LLM serve text models. The project is part of a broader trend toward specialized infrastructure for next-generation AI workloads.