Running Pods keep running when a Kubernetes node loses its connection to the control plane. The kubelet on each node keeps its containers alive, restarts them when they crash, and keeps running their health checks without asking the API server. What stops is everything that needs a decision from the control plane: new deployments, scaling, rescheduling, and any workload that talks to the Kubernetes API itself.
That answer matters most to teams that run Kubernetes nodes far from the control plane: in stores, factories, ships, branch offices, or on-premises data centers connected over the internet. This post explains what happens step by step, where the defaults work against you at edge sites, and how to design workloads that ride out a disconnect.
Two kinds of disconnect
A node can lose the control plane in two ways, and Kubernetes reacts differently to each.
In a control plane outage, the API server is unavailable for every node at once, for example during maintenance or a failure on the control plane side. The controllers that would react to the outage are part of the control plane, so they are down too. Nothing gets evicted or rescheduled. Workloads keep running exactly as they were, and the cluster resumes where it left off when the control plane returns.
In a site disconnect, the control plane is healthy, but one location loses its uplink. The control plane sees those nodes go silent while the rest of the cluster reports normally, and it starts to act on what it sees. This is the case that needs planning.
What happens during a site disconnect
| Time after the link drops | What happens |
|---|---|
| Immediately | Pods on the site keep running. The kubelet keeps restarting crashed containers and running liveness and readiness probes locally. Traffic between Pods on the site continues. |
| Under a minute | The node stops renewing its lease. The control plane marks the node NotReady with status Unknown and adds the node.kubernetes.io/unreachable taint. |
| About 5 minutes | The default toleration for the unreachable taint expires. The control plane marks the site’s Pods for deletion and creates replacements for Deployment and ReplicaSet Pods on nodes it can still reach. |
| Until the link returns | The kubelet on the site never receives the deletion, so the original Pods keep running there. StatefulSet Pods are not replaced, because Kubernetes cannot confirm the old Pod has stopped. |
| When the link returns | The node renews its lease and becomes Ready. The kubelet learns which of its Pods were deleted and stops them, and the cluster converges on the state stored in the control plane. |
The five-minute step is where edge sites get into trouble. Kubernetes assumes an unreachable node is dead and that its work should move elsewhere. For a cloud node in a failed zone, that is the right call. For a store whose uplink dropped, it means the control plane starts replacement Pods in another location, and when the link returns, it stops the local Pods that served customers the whole time.
What keeps working and what stops
Everything the node can do on its own continues: running containers, restarting them according to their restart policy, probes, local volumes, and network traffic between Pods that already know about each other. Containers whose images are cached on the node restart without a registry, as long as the image pull policy allows it.
Everything that depends on the API server pauses until the link returns. You cannot deploy, scale, or roll back with kubectl. The Horizontal Pod Autoscaler does not add replicas, CronJobs do not start, and changes to ConfigMaps and Secrets do not reach the site. Service endpoints stop updating, so new Pods do not receive traffic and removed Pods are not taken out of rotation. Operators and other workloads that call the Kubernetes API fail their requests, and anything that relies on Kubernetes leases for leader election loses its leadership when it cannot renew the lease.
Designing workloads for disconnected sites
A few settings decide whether a disconnect is a non-event or an outage.
Keep site workloads at the site. Pin workloads that serve a location to that location’s nodes with a nodeSelector or node affinity, for example on a site label. If the control plane creates replacement Pods during a disconnect, they then stay Pending instead of starting somewhere they should not run.
Tolerate the unreachable taint for longer. The five-minute default comes from a toleration that Kubernetes adds to every Pod. Set your own toleration to keep site workloads bound to their nodes through longer outages:
tolerations:
- key: node.kubernetes.io/unreachable
operator: Exists
effect: NoExecute
tolerationSeconds: 86400
- key: node.kubernetes.io/not-ready
operator: Exists
effect: NoExecute
tolerationSeconds: 86400
Leave out tolerationSeconds and the Pods are never evicted for an unreachable node. That suits workloads that exist only at one site, but not services that should fail over to another location.
Make restarts independent of the network. Use image tags or digests that are already on the node with imagePullPolicy: IfNotPresent, or run a registry mirror at the site. Keep the data a workload needs at startup on local storage.
Avoid control plane dependencies in the request path. An application that reads its configuration from the Kubernetes API at startup, or elects a leader through a Kubernetes lease, fails during a disconnect even though its node is healthy. Read configuration from mounted files and choose leader election that works within the site, or run a single replica where that is acceptable.
Keep dependencies at the site. A Pod that calls a database, cache, or API that runs only in the cloud depends on the link as much as on its own node. Run the services a site needs at that site, and make sure cluster DNS answers there too.
Control plane availability still matters
None of this makes the control plane optional. Every minute it is unreachable is a minute without deployments, scaling, and self-healing beyond the node. A highly available control plane shrinks the first kind of disconnect to rare events, so the design work above only needs to cover site links.
Cloudfleet runs the control plane as a managed service. Pro clusters replicate it across availability zones on dedicated resources with a 99.95% monthly uptime SLA, and Enterprise clusters raise that to 99.99% with custom terms. See cluster types for the details, and cannot reach the cluster API for what to check during an interruption.
Edge and on-premises servers join a Cloudfleet cluster as self-managed nodes with outbound connections only. They work behind NAT and need no inbound firewall rules, so a site needs nothing more than an internet uplink. When that uplink drops, the site keeps running its Pods, cluster DNS keeps resolving names, and Pods on the site keep talking to each other over the encrypted network between nodes. The cluster picks up changes as soon as the uplink returns. One cluster spans all your sites and clouds, without a separate cluster to operate at every location.
Teams coming from Google Distributed Cloud connected, formerly Google Distributed Cloud edge, often look for exactly this setup on their own hardware. The Google Distributed Cloud and Anthos alternative page compares the two, and what happened to Google Anthos explains how Google’s edge editions fit together.
Frequently asked questions
Do Pods keep running if the Kubernetes control plane goes down?
Yes. The kubelet on each node keeps running its Pods, restarts crashed containers, and runs health checks without the control plane. Deployments, scaling, and rescheduling stop until the control plane is back.
What happens to a Kubernetes node that loses network connectivity?
After less than a minute, the control plane marks the node NotReady and taints it as unreachable. After about five minutes, it marks the node’s Pods for deletion and creates replacements elsewhere. The Pods keep running on the node until it reconnects and learns they were deleted.
How do I stop Kubernetes from evicting Pods at an edge site that goes offline?
Add a toleration for the node.kubernetes.io/unreachable taint with a long tolerationSeconds, or none for no eviction at all, and pin site workloads to the site’s nodes so replacements cannot start elsewhere.
Can a Kubernetes edge site run fully offline?
For hours or days, yes, if its workloads keep their images, data, and dependencies at the site. Changes to the site wait until the connection returns. Sites that must run permanently without a connection need a control plane at the site.
