Why is it hard to get GPU capacity in the cloud?
Getting cloud GPUs is hard because three separate limits stand in the way: quota, capacity and allocation. Quota is your account’s permission to run GPUs, and new accounts often start at zero. Capacity is the physical stock of a GPU type in a zone, which runs out regardless of your quota. Allocation is the share of new GPUs that providers sell to committed customers before anything reaches on-demand.
Quota is your account’s permission
Every large cloud caps how many GPUs an account may run. The cap exists to limit fraud and to ration scarce hardware, and for GPUs it often starts at zero.
- On AWS, on-demand quotas are counted in vCPUs per instance family. As of November 2026, the default for both Running On-Demand P instances and Running On-Demand G and VT instances is 0. A single 8-GPU
p5.48xlargeneeds 192 vCPUs of P quota. - On Google Cloud, GPU quota is set per GPU model and region, plus a global
GPUs (all regions)quota. Spot, flex-start and other preemptible GPUs draw on a separate preemptible GPU quota. - On Azure, quota is set per VM family and region, and Microsoft’s documentation states that quota and capacity are separate checks.
Quota requests are reviewed, sometimes by a person. Requests that name a region, a GPU type, a realistic count and the workload are approved more often than a request for 64 H100s on a week-old account. Starting small and raising the quota as billing history builds up is the usual path.
Capacity is hardware in the zone
Quota is permission, not hardware. Once you have quota, the request can still fail because the zone has no free GPUs of that type. Each cloud has its own error for it:
| Cloud | Quota error | Capacity error |
|---|---|---|
| AWS | VcpuLimitExceeded or InstanceLimitExceeded | InsufficientInstanceCapacity |
| Google Cloud | QUOTA_EXCEEDED | ZONE_RESOURCE_POOL_EXHAUSTED and its _WITH_DETAILS variant |
| Azure | OperationNotAllowed, “exceeding quota limits” | AllocationFailed or ZonalAllocationFailed |
Google’s documentation is explicit that a capacity error reflects a hardware shortage in the zone and that a quota increase does not resolve it. The standard advice from all three is the same: retry later, try another zone, request fewer instances at a time, or pick a different instance type.
Allocation decides who gets new GPUs first
Behind quota and capacity sits a supply question. Most new data center GPUs, and almost all rack-scale systems such as GB200 and GB300 NVL72, are sold as reservations and committed contracts rather than on demand. On-demand pools are what is left after those commitments, which is why a GPU can appear on a pricing page and still be unavailable in every zone you try. The buying models, from on-demand to multi-year contracts, are compared in GPU procurement in a shortage.
Products that guarantee capacity
Hyperscalers sell products that trade flexibility for a guarantee:
| Product | What it guarantees | Limits |
|---|---|---|
| AWS Capacity Blocks for ML | GPU instances for a fixed future window, booked up to 8 weeks ahead | Up to 64 instances per block and 256 across blocks; no cancellation |
| Google Cloud flex-start | VMs once capacity frees up, for up to 7 days | You wait until capacity is available; uses preemptible GPU quota |
| Google Cloud calendar mode | A reservation for a future period of up to 90 days | Future availability is visible up to 60 days ahead for GPU VMs |
| Azure on-demand capacity reservations | Capacity in a region or zone until you delete it | Billed while unused; as of November 2026 not available for the ND series or NCads H100 v5 |
These help when you can plan. They do not help a team that needs an H100 this afternoon in a region that has none.
What to do when you cannot get GPUs
- Spread across zones and regions. A GPU type sold out in one zone is often available in another. Inference that serves EU users can still run in two or three EU regions.
- Use more than one provider. Specialized GPU clouds, often called neoclouds, list H100, H200 and B200 capacity that hyperscalers do not have on hand. For EU-only options, see where to rent GPUs in Europe.
- Step down a GPU generation. Many inference and fine-tuning jobs run fine on A100, L40S or L4, which are easier to find than the newest parts. Check whether the model fits first; how much GPU memory an LLM needs has the math.
- Use spot for interruptible work. Training with frequent checkpoints can run on spot GPUs, which draw on a separate quota and a different pool. See what spot instances are.
- Reserve the baseline. If you need the same GPUs every day, a reservation or committed contract is the only reliable way to have them.
Spreading across providers solves capacity but creates an operations problem: each provider brings its own API, images, networking and quota process. Running all of them as one Kubernetes cluster keeps one way to deploy, as discussed in whether a Kubernetes cluster can span multiple clouds.
Finding GPU capacity with Cloudfleet
Cloudfleet runs GPU nodes from AWS, Google Cloud, Azure, Hetzner, OVHcloud, Scaleway, Lambda, Nebius and your own servers in one managed Kubernetes cluster, so workloads can land wherever GPUs are available. The Cloudfleet team also helps source reserved and committed GPU capacity, which you contract and pay for directly with the provider.