Our Kubernetes Operator Didn't Scale, So We Rebuilt It
Infisical rebuilt their Kubernetes Operator from scratch because the original design had scalability and performance issues. The new operator improves how secrets are synced and managed across Kubernetes clusters, addressing the limitations that emerged as usage grew.
Background
Infisical is a startup that provides an open-source platform for managing secrets (API keys, database passwords, certificates) used by software applications. Their first Kubernetes operator — a program that extends Kubernetes to automate tasks, in this case syncing secrets into the cluster — was written in Python and handled each secret one at a time, which broke down at hundreds of secrets. The rebuild in Go uses a controller-runtime pattern that can watch and reconcile all secrets in parallel, and handles GitOps workflows (automated syncs triggered by changes in a Git repo) without the bottleneck of a sidecar per pod. The article is a technical post-mortem explaining why the old architecture failed and how the new one solves it, relevant to any team running Kubernetes who need a scalable way to inject dynamic secrets into their infrastructure.