How does Kubernetes autoscaling work?

Kubernetes autoscaling works at two levels. Workload autoscalers change the Pods: the Horizontal Pod Autoscaler (HPA) changes how many replicas run, and the Vertical Pod Autoscaler (VPA) changes how much CPU and memory each one requests. A node autoscaler then changes the cluster itself, adding nodes when Pods cannot be scheduled and removing nodes that are no longer needed.

The autoscaling layers at a glance

Horizontal Pod autoscaling

The HPA is a controller in kube-controller-manager. Every 15 seconds by default it reads a metric, such as average CPU utilization across a Deployment’s Pods, and calculates the desired replica count:

desiredReplicas = ceil(currentReplicas × currentMetricValue / desiredMetricValue)

Ten Pods at 90% CPU against a 60% target become 15 Pods. The HPA ignores changes within a 10% tolerance to avoid flapping. By default it scales up immediately, by up to 100% or 4 Pods every 15 seconds, whichever is more, and it waits for a 300-second stabilization window before scaling down.

Utilization targets depend on resource requests. The documentation is explicit: if a container in the Pod has no request for the metric, “CPU utilization for the Pod will not be defined and the autoscaler will not take any action for that metric”. Resource metrics also need the Metrics Server running in the cluster. See resource requests and limits for how to set them.

For load that CPU does not reflect, such as messages waiting in a queue, the CNCF project KEDA feeds external event sources into horizontal scaling and can scale a workload to zero. Its Cron scaler also covers schedule-based scaling. Upstream Kubernetes added its own scale-to-zero support behind the HPAScaleToZero feature gate, beta and enabled by default since Kubernetes 1.37, for object and external metrics only.

Vertical Pod autoscaling

The VPA is an add-on with three parts: a recommender that analyzes usage, an updater that applies new requests and an admission controller that sets requests on new Pods. Its update mode decides how far it goes:

The older Auto mode is deprecated since VPA 1.4.0 and behaves like Recreate. In-place resizing builds on the Kubernetes in-place Pod resize feature, stable since Kubernetes 1.35, which changes a running container’s CPU and memory without recreating the Pod.

Node autoscaling

Workload autoscalers only create or resize Pods. When the new Pods do not fit on existing nodes, they stay Pending, and a node autoscaler has to add capacity. According to the Kubernetes documentation, node autoscalers “either choose them from a pre-defined set of Node configurations, or use auto-provisioning”. These are the two models:

The upstream Cluster Autoscaler checks for unschedulable Pods every 10 seconds. It considers a node for removal when the requests of its Pods use less than 50% of its capacity, and removes it after it has been unneeded for 10 minutes. It does not support auto-provisioning: every node group has to exist in advance.

Both models decide based on resource requests, not real usage. Requests that are too low lead to new nodes that still cannot run the Pod well. Requests that are too high block consolidation and leave nodes half empty.

How the layers chain together

1. Traffic rises and average CPU across a Deployment passes the HPA target.
2. The HPA raises the replica count, and the Deployment creates new Pods.
3. The scheduler finds no node with enough free CPU and memory, so the new Pods stay Pending.
4. The node autoscaler sees the pending Pods and provisions one or more nodes.
5. The new nodes register with the control plane, and the scheduler places the Pods on them.

When traffic falls, the chain runs backwards: the HPA removes replicas, nodes become underused and the node autoscaler drains and deletes them. New nodes take time to boot and join, so step 4 adds a delay that HPA settings alone cannot remove.

Scale-down and disruption

Removing a node evicts its Pods. PodDisruptionBudgets (PDBs) limit how many replicas of a workload can be down at once, and the Cluster Autoscaler checks that “PodDisruptionBudgets for pods scheduled there allow for removing at least one replica” before it terminates a node. Its FAQ also lists Pods that block removal, including Pods with restrictive PDBs, Pods not managed by a controller, Pods with local storage and Pods annotated as not safe to evict.

Common failure modes

• No resource requests. The HPA cannot compute utilization and does nothing.
• HPA and VPA on the same metric. The VPA known limitations say VPA “should not be used with the Horizontal Pod Autoscaler (HPA) on the same resource metric (CPU or memory)”. Use HPA on CPU and VPA on memory, or HPA on custom metrics.
• VPA on DaemonSets with node autoscaling. The Kubernetes documentation advises against it, because node autoscalers predict DaemonSet requests on new nodes.
• Restrictive PDBs. A PDB that allows zero disruptions keeps nodes alive indefinitely.
• Low maxReplicas or node limits. Autoscaling stops at the ceiling you set, and Pods stay Pending.

Autoscaling on Cloudfleet

Cloudfleet Kubernetes Engine (CFKE) includes the Metrics Server for HPA and has built-in node auto-provisioning on AWS, GCP and Hetzner Cloud, which picks the most cost-effective instance type for pending Pods and removes nodes that are no longer needed. See node auto-provisioning and Pod autoscaling in the documentation.