What are hosted control planes in Kubernetes?
A hosted control plane is a Kubernetes control plane that runs as ordinary Pods inside a separate management cluster, instead of on dedicated control plane machines next to its worker nodes. Each hosted cluster keeps its own API server, controllers and datastore, but they share the management cluster’s hardware, and its worker nodes can run in another network, cloud or data center. Kamaji, HyperShift, k0smotron and Gardener are open-source projects built on this model.
How it works
In a classic cluster, three or more virtual machines run kube-apiserver, etcd, the scheduler and the controller manager, and the worker nodes sit beside them. In the hosted model, those components are Deployments and StatefulSets in a management cluster, one set per tenant cluster:
- Management cluster. A regular Kubernetes cluster that runs the control planes of many other clusters and an operator that creates, scales and upgrades them.
- Tenant or hosted cluster. The cluster your users see. Its API server is exposed through a load balancer or ingress on the management cluster.
- Worker nodes. Machines that join the hosted cluster’s API server over the network. They do not need to be in the same cloud or data center as the management cluster.
- Datastore. Each hosted control plane needs etcd or an etcd-compatible store. Some projects give each cluster its own etcd, others can share a database between clusters.
If you need a refresher on the components themselves, see what the Kubernetes control plane is.
Projects that implement hosted control planes
| Project | Description |
|---|---|
| Kamaji | An operator that runs tenant control planes as Pods in a management cluster, built from unmodified upstream components and kubeadm. A datastore can be dedicated to one cluster or shared, and supports etcd, or MySQL and PostgreSQL through kine. |
| HyperShift | Red Hat’s middleware for hosting OpenShift control planes at scale, sold as hosted control planes for OpenShift. |
| k0smotron | An operator that runs k0s control planes inside an existing cluster and integrates with Cluster API. |
| Gardener | SAP’s project with garden, seed and shoot clusters. Seeds host the control planes of many shoot clusters as Pods, and a shoot consists only of worker nodes. |
Several of these projects plug into Cluster API, which handles creating and replacing the worker machines.
Hosted control planes vs virtual clusters
Virtual clusters, such as vCluster, also run a control plane as a Pod inside another cluster. The difference is where workloads run. In vCluster’s default shared-nodes mode, tenant Pods run on the host cluster’s own nodes, and vCluster states that this mode is not a security boundary for untrusted tenants. A hosted control plane always has its own worker nodes, so workloads of different clusters never share a kernel. vCluster’s licensed private-nodes mode moves it closer to the hosted model. Kubernetes multi-tenancy compares these isolation levels.
Benefits
- Lower cost per cluster. Small clusters do not need three dedicated control plane machines each. Many control planes share one management cluster, packed by resource requests like any other workload.
- Faster provisioning. Creating a cluster means creating Pods, not booting and configuring virtual machines.
- Central operations. Upgrades, backups, certificates and monitoring for all control planes happen in one place, using Kubernetes itself.
- Separation of concerns. The team that runs control planes does not need access to tenant nodes, and tenants cannot touch control plane machines.
- Nodes anywhere. Because nodes only need to reach the API server, one management cluster can serve clusters whose nodes run in different clouds, data centers or edge sites.
Trade-offs
- Control plane to node connectivity. Kubernetes traffic does not only flow from nodes to the API server. The API server also connects to kubelets for
kubectl logs,execandport-forward, and to webhooks and metrics endpoints in the cluster network. When nodes sit behind NAT or firewalls, a tunnel is needed. The upstream answer is Konnectivity, a TCP-level proxy for control-plane-to-cluster traffic. Kamaji uses agents on worker nodes that open outbound connections to a Konnectivity server, and Gardener uses a VPN between seed and shoot. - Latency. Nodes far from the management cluster see slower API calls and slower reactions to node failure. Choose a management cluster region close to most nodes.
- Blast radius. A failure of the management cluster, its network or a shared datastore affects every hosted control plane at once. Running workloads keep running, as they do whenever a control plane is unavailable, but no cluster can schedule or scale until it recovers.
- Noisy neighbors. One tenant’s heavy API traffic can compete for resources with others on the same management cluster unless requests, limits and API priority settings isolate them.
- More components to operate. The management cluster itself needs upgrades, backups and capacity planning.
Where hosted control planes are used
The model fits anyone who runs many clusters for others: internal platform teams giving each team its own cluster, software vendors running a cluster per customer, and providers offering managed Kubernetes. Do not assume a specific managed service uses it unless the provider documents it. For a broader view of who runs control planes and what you give up by running your own, see managed vs self-managed Kubernetes.
Control planes on Cloudfleet
Cloudfleet runs an isolated control plane and cluster data store for every cluster in a control plane region you choose, and nodes join it from any cloud provider or your own data center. Pro and Enterprise clusters run the control plane on dedicated resources, replicated across availability zones. See cluster types.