Skip to content
TopicTracker
From HackerNewsView original
TranslationTranslation

The Qbeast split-plane SaaS Architecture

Qbeast introduces a split-plane SaaS architecture that separates data plane from control plane, enabling multi-tenant isolation and scalability. This design allows independent scaling of compute and storage resources while maintaining strict tenant boundaries.

Background

- Qbeast is an open-source data lake engine designed to improve the performance and manageability of big data stored in cloud object stores (like AWS S3). It competes with file formats like Apache Parquet and Apache Iceberg. - "Split-plane" is Qbeast's architectural pattern for SaaS (software-as-a-service) multi-tenancy: it separates the *control plane* (metadata, indexing, schema management) from the *data plane* (actual storage and query execution). - This split lets Qbeast's indexing and compaction happen independently on the control side, while the data plane remains a plain object store. Tenants (different customers) can share the same storage infrastructure without interfering with each other's write/read performance or indexing schedules. - The key idea: by decoupling metadata operations from data I/O, Qbeast avoids bottlenecks common in monolithic lakehouse architectures, where one heavy job (like compaction) can block queries across all tenants.

Related stories