How do LoadBalancer Services work on bare-metal Kubernetes?
On bare-metal and on-premises Kubernetes, a Service of type LoadBalancer gets an external IP only after you install a load balancer implementation, because Kubernetes does not include one. Tools such as MetalLB, Cilium and kube-vip fill the gap: they assign an IP address from a pool you define and announce it to your network, either with Layer 2 (ARP or NDP) from one node at a time, or with BGP to your routers, which spread traffic across several nodes.
Why EXTERNAL-IP stays pending
The Kubernetes documentation is direct about it: “Kubernetes does not directly offer a load balancing component; you must provide one, or you can integrate your Kubernetes cluster with a cloud provider.” In a cloud, the cloud controller manager watches for LoadBalancer Services and creates a load balancer through the provider’s API. On your own hardware there is no such API, so the Service shows <pending> in the EXTERNAL-IP column indefinitely.
The built-in fallbacks have limits. A NodePort Service opens a port from the 30000 to 32767 range on every node, so clients need to know node addresses and a high port. externalIPs routes an address you assign by hand, with no allocation and no failover. Neither gives you a stable address on a standard port that survives a node failure.
Two jobs: assigning and announcing IPs
A bare-metal load balancer does two separate things:
- IP address management. It hands out addresses from a range you reserve, such as part of a data center subnet that your DHCP server does not use. In MetalLB this is an
IPAddressPool; in Cilium it is LB IPAM, which “only does IP allocation”. - Announcement. It tells the network which node to send traffic for that address to. This is where Layer 2 mode and BGP mode differ.
Once traffic reaches a node, kube-proxy or the CNI plugin forwards it to a backend Pod, as with any Service.
Layer 2 vs BGP
| Aspect | Layer 2 mode (ARP or NDP) | BGP mode |
|---|---|---|
| How it works | One elected node answers ARP (IPv4) or NDP (IPv6) requests for the service IP | Nodes open BGP sessions with your routers and advertise a route to the service IP |
| Traffic distribution | All traffic for a service enters through one node | Routers spread connections across nodes with ECMP |
| Bandwidth limit | One node’s network bandwidth per service | Combined bandwidth of the advertising nodes |
| Network requirements | Nodes and clients on the same Layer 2 segment; no router changes | BGP-capable routers, an AS number for the cluster, peering configuration |
| Failover | Another node takes over the IP and sends gratuitous ARP | The router withdraws the route when a node’s BGP session drops |
| Effect of a failover | Short interruption while clients update their ARP cache | Connections through the failed node are reset; a change in node count can move existing connections |
| Fits | Labs, small sites, flat networks | Production data centers, multiple racks, high throughput |
Failover times depend on the implementation. MetalLB’s Layer 2 mode typically fails over within a few seconds on modern operating systems and treats more than about 10 seconds as abnormal. Cilium’s L2 announcements, a beta feature, use Kubernetes leases with a default duration of 15 seconds, so failover takes between 10 and 20 seconds with default settings.
In MetalLB’s BGP mode, routers hash each connection to one node. When a node leaves, active connections through it fail with resets, and routers with unstable hashing can move other connections too. Resilient ECMP on the router, an ingress controller in front of applications and client retries reduce the impact.
The tools
| Tool | Modes | Notes |
|---|---|---|
| MetalLB | Layer 2, BGP | Works with any CNI plugin. FRR-K8s is the default BGP backend. |
| Cilium | LB IPAM with BGP control plane or L2 announcements | Built into the CNI plugin, so no extra component if Cilium already runs the cluster network. L2 announcements require kube-proxy replacement. |
| kube-vip | ARP, BGP | Provides a virtual IP for the control plane API endpoint as well as for LoadBalancer Services. |
| Hardware or virtual appliances | Vendor-specific | F5 BIG-IP and similar appliances integrate through the vendor’s Kubernetes controller. |
MetalLB vs Cilium usually comes down to the CNI plugin. If the cluster runs Cilium, its BGP control plane replaces MetalLB’s BGP mode without another component. If it runs another CNI, MetalLB is the common choice. Do not run two implementations that announce the same address range. What a CNI plugin does explains how the choice affects networking beyond load balancing.
The API server needs a stable address too. A highly available control plane puts a virtual IP in front of its API servers, and the kubeadm project documents keepalived with HAProxy, and kube-vip for this.
Setup checklist
- Reserve an IP range for load balancer Services and exclude it from DHCP.
- For BGP, agree on AS numbers with the network team, configure the routers to accept the routes, and enable ECMP.
- Decide on
externalTrafficPolicy.Localpreserves the client source IP and sends traffic only to nodes that run a backend Pod;Clusterspreads traffic to all nodes and adds a hop. - Put an ingress or Gateway API controller behind one load balancer IP for HTTP services, instead of one IP per application.
- Test failover by draining the announcing node and measuring the interruption.
Load balancing is one of several components an on-premises cluster has to provide itself; running Kubernetes on-premises lists the rest.
How Cloudfleet fits
Cloudfleet Kubernetes Engine (CFKE) uses Cilium as its CNI plugin and supports Cilium’s BGP advertisement for LoadBalancer Services on self-managed nodes, with separate BGP configuration per site through node labels. MetalLB also works for teams that already use it. See on-premises load balancing with BGP.