NVIDIA Enterprise Reference Architectures are NVIDIA’s validated blueprints for building an AI factory: a GPU cluster of 4 to 128 nodes, designed as a whole from servers and network fabric up to the software that schedules work on it. Each design specifies how many CPUs, GPUs, and network adapters a node has, how much bandwidth each GPU gets, how nodes connect as the cluster grows, and which software runs on top. The designs support industry-standard orchestration, including Kubernetes and Slurm.
For a Kubernetes team, the reference architectures answer two practical questions: what the hardware underneath your GPU nodes should look like, and which NVIDIA components to run in the cluster. This post covers both.
How to read 2-8-5-200
Every Enterprise RA configuration has a four-part name in the form CPUs, GPUs, network adapters, and east-west bandwidth per GPU. A 2-8-5-200 node has 2 CPUs, 8 GPUs, and 5 network adapters, with 200 Gb per second of east-west bandwidth for each GPU. Of the 5 adapters, 4 carry east-west traffic between GPU nodes, and 1 carries north-south traffic to storage, users, and the rest of the data center.
The ratio is the point. East-west bandwidth is what multi-node training and large-model inference use to exchange data between GPUs, so the designs give every GPU, or every pair of GPUs, a dedicated path into the compute fabric.
The three AI factory families
NVIDIA groups the reference architectures into three families, each built from scalable units that grow in fixed steps:
| Family | Foundational configuration | GPUs | Scale | Networking |
|---|---|---|---|---|
| RTX PRO AI Factory | 2-8-5-200 | RTX PRO 6000 or RTX PRO 4500 Blackwell Server Edition | 4 to 32 nodes in steps of 4, 32 to 256 GPUs | BlueField-3 for east-west and north-south |
| HGX AI Factory | 2-8-9-800 | HGX B300 | 4 to 128 nodes in steps of 4, 32 to 1,024 GPUs | 8 ConnectX-8 SuperNICs east-west, BlueField-3 north-south |
| NVL72 AI Factory | 2-4-5-800 | GB300 NVL72 | 1 to 8 racks, 72 to 576 GPUs | 4 ConnectX-8 east-west, BlueField-3 north-south |
The RTX PRO family also has smaller volume configurations, 2-4-3-200 and 2-2-3-400, for workloads that need some east-west networking but not eight GPUs per node, and extreme volume configurations without an east-west fabric. In the RTX PRO designs, east-west and north-south traffic can share one fabric up to 16 nodes, and larger clusters get dedicated networks.
The software layer on Kubernetes
Above the hardware, the reference architectures build on NVIDIA AI Enterprise and a set of Kubernetes operators. Each handles one layer:
| Component | Layer | What it does |
|---|---|---|
| NVIDIA GPU Operator | GPU nodes | Installs and manages the driver, container toolkit, device plugin, GPU Feature Discovery, DCGM Exporter for telemetry, MIG Manager, and a DRA driver |
| NVIDIA Network Operator | Network fabric | Deploys DOCA drivers for ConnectX and BlueField adapters, secondary networks through Multus and IPAM, and device plugins for RDMA and SR-IOV interfaces |
| NVIDIA NIM Operator | Inference | Caches model artifacts (NIMCache), runs NIM inference microservices (NIMService), and groups them into pipelines (NIMPipeline) |
| NVIDIA Run:ai | Scheduling | Guaranteed GPU quotas per team, queueing, priorities and preemption, fractional GPUs, and topology-aware placement |
| Base Command Manager | Cluster provisioning | Provisions, monitors, and scales the cluster, based on upstream Kubernetes |
The GPU Operator and Network Operator work together. GPUDirect RDMA, which lets GPUs on different nodes exchange data through the network adapters without going through the CPU, needs both: the GPU Operator for the GPU side and the Network Operator for the adapters and secondary networks. Without the Network Operator, Pods only see the cluster’s standard overlay network, and multi-node jobs cannot use the east-west fabric that the hardware design paid for.
Run:ai’s scheduler is also available as open source. In April 2025, NVIDIA released it as KAI Scheduler under the Apache 2.0 license, with gang scheduling, GPU sharing, and hierarchical queues.
Installing the operators on a cluster that already has drivers
Many Kubernetes platforms install the NVIDIA driver and container toolkit on GPU nodes themselves. The GPU Operator supports this directly: disable the components that are already present, and it manages the rest.
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install gpu-operator nvidia/gpu-operator \
--namespace gpu-operator --create-namespace \
--set driver.enabled=false \
--set toolkit.enabled=false
Both operators also deploy Node Feature Discovery by default. If it already runs in your cluster, set nfd.enabled=false so you do not run it twice.
What a managed platform covers, and what you add
Cloudfleet follows the same layered design with standard Kubernetes. Cloudfleet installs the NVIDIA driver and container toolkit on every GPU node it provisions in the cloud, and on your own servers when they join with the --install-nvidia-drivers flag. The node auto-provisioner adds GPU nodes when Pods request nvidia.com/gpu and labels them with cfke.io/accelerator-name and cfke.io/accelerator-memory. Time slicing and MPS let inference workloads share GPUs on the Pro and Enterprise tiers. The GPU workloads documentation covers the details.
On top, you deploy the NVIDIA components your workloads need with standard Helm charts: DCGM Exporter for GPU telemetry, the Network Operator on bare-metal nodes with ConnectX or BlueField adapters, the NIM Operator for NIM inference services, and Run:ai or KAI Scheduler for team quotas and gang scheduling. Nothing proprietary sits between them and the Kubernetes API.
The difference from a self-built AI factory is the cluster itself. Cloudfleet runs the control plane, and one cluster can span GPU nodes from several providers and your own servers, so a training Fleet on HGX nodes and an inference Fleet on RTX PRO or L4 nodes can share images, pipelines, and monitoring. See training and inference in one Kubernetes cluster for that setup, the GPU Kubernetes page for available capacity, and our comparison of NVIDIA GPU architectures for background on the GPU generations.
Frequently asked questions
What are NVIDIA Enterprise Reference Architectures?
They are NVIDIA’s validated designs for AI factories from 4 to 128 nodes. Each specifies the server configuration, the network fabric and its scaling steps, and the software stack, so that partners and customers can build GPU clusters that perform as tested.
What does 2-8-5-200 mean?
It describes a node with 2 CPUs, 8 GPUs, and 5 network adapters, with 200 Gb per second of east-west bandwidth per GPU. Four adapters connect the GPUs to the compute fabric, and one connects the node to storage and the data center network.
What is the NVIDIA GPU Operator?
A Kubernetes operator that manages the NVIDIA software GPU nodes need: the driver, container toolkit, device plugin, GPU Feature Discovery, DCGM Exporter, MIG Manager, and a DRA driver. Components that the platform already installs can be disabled with Helm values.
What is the NVIDIA Network Operator for?
It prepares the network fabric for GPU workloads: drivers for ConnectX and BlueField adapters, secondary networks through Multus, and device plugins for RDMA and SR-IOV. Together with the GPU Operator, it enables GPUDirect RDMA between nodes.
What is an NVIDIA AI factory?
NVIDIA’s term for a data center built to produce AI, from training models to serving them. The Enterprise Reference Architectures describe AI factories in three families: RTX PRO, HGX, and GB300 NVL72.
