Resources

Terraform resources in the Cloudfleet provider allow you to define and manage your infrastructure as code. You can create and configure Kubernetes clusters, set up multi-cloud fleets with automatic node provisioning, and generate node join instructions for any environment - including on-premises and unsupported clouds. These resources are essential for provisioning and scaling Cloudfleet environments consistently and repeatably.

cloudfleet_cfke_cluster

Creates and manages Cloudfleet Kubernetes Engine (CFKE) clusters.

Required arguments:

name - Cluster name (1-63 characters, must match ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$)
region - Cloudfleet control plane region (for example, northamerica-central-1a or europe-central-1a). Immutable; changing it replaces the cluster.

Optional arguments:

tier - Cluster tier (basic, pro, or enterprise). Updated in place.
version_channel - Kubernetes version channel (defaults to 1.x.x-cfke.x)
release_channel - Release channel for the control plane (stable, rapid, or extended, defaults to rapid). Updated in place.
features - Cluster feature toggles, updated in place. When the block is set, gpu_sharing_strategy and gpu_max_shared_clients_per_gpu are required.
- gpu_sharing_strategy - GPU sharing strategy (none, mps, or time_slicing)
- gpu_max_shared_clients_per_gpu - Maximum number of pods that may share a single GPU (2-48)
- cilium_socket_lb_host_namespace_only - Restrict Cilium socket load-balancing to the host namespace (required for the Tailscale operator and Istio)
networking - Cluster networking configuration. Immutable; changing it replaces the cluster.
- pod_cidr - CIDR block for pod IPs
- service_cidr - CIDR block for service IPs
- dual_stack - Enable IPv4+IPv6 dual-stack networking
- pod_cidr_v6 - IPv6 pod CIDR (requires dual_stack)
- service_cidr_v6 - IPv6 service CIDR (requires dual_stack)

Read-only attributes:

id - Cluster identifier
endpoint - Kubernetes API endpoint
certificate_authority - Base64 encoded CA certificate
kubernetes_version - Current Kubernetes version
status - Cluster status
created_at - Creation timestamp
updated_at - Last update timestamp

The mutable scalar fields (tier, version_channel, release_channel) and the features block are applied with full-overwrite semantics: omitting a field on update resets it to its default rather than leaving it unchanged. release_channel, version_channel, features, and custom networking are Pro and Enterprise tier features; Basic clusters use the platform defaults.

Example:

terraform

# Basic cluster
resource "cloudfleet_cfke_cluster" "basic" {
  name   = "my-basic-cluster"
  region = "europe-central-1a"
  tier   = "basic"
}

# Cluster with a release channel, GPU sharing, and custom networking
resource "cloudfleet_cfke_cluster" "advanced" {
  name            = "my-advanced-cluster"
  region          = "europe-central-1a"
  tier            = "pro"
  release_channel = "stable" # stable | rapid | extended

  # Feature toggles, updated in place
  features {
    gpu_sharing_strategy           = "time_slicing" # none | mps | time_slicing
    gpu_max_shared_clients_per_gpu = 4
  }

  # Networking is immutable: changing it replaces the cluster
  networking {
    pod_cidr     = "10.96.0.0/16"
    service_cidr = "10.97.0.0/16"
  }
}

# Pro cluster with specific Kubernetes version
resource "cloudfleet_cfke_cluster" "pro" {
  name            = "my-pro-cluster"
  region          = "northamerica-central-1a"
  tier            = "pro"
  version_channel = "1.33.x-cfke.x"
}

cloudfleet_cfke_fleet

Creates and manages multi-cloud infrastructure fleets for CFKE clusters. Fleets enable automatic node provisioning through the node auto-provisioner for supported cloud providers.

Required arguments:

name - Fleet name. Immutable; changing it replaces the fleet.
cluster_id - Cluster identifier this fleet belongs to. Immutable; changing it replaces the fleet.

Optional arguments:

scaling_profile - Controls scale-up/scale-down responsiveness (aggressive for cost optimization or conservative for stability, defaults to conservative). Updated in place.
limits - Fleet resource limits, updated in place
- cpu - CPU limit in cores (omit to place no CPU bound on the fleet)
constraints - Scheduling and provisioning constraints, updated in place. Each set restricts the allowed values for a Kubernetes label key; omit a set for no restriction.
- arch - Allowed values for kubernetes.io/arch (for example, ["amd64", "arm64"])
- capacity_type - Allowed values for karpenter.sh/capacity-type (for example, ["on-demand", "spot"])
- instance_family - Allowed values for cfke.io/instance-family (validated by the API)
- region - Allowed values for topology.kubernetes.io/region (validated by the API)
aws - AWS fleet configuration, updated in place
- role_arn - AWS IAM role ARN for Karpenter resource management
gcp - GCP fleet configuration, updated in place
- project_id - GCP Project ID for instance deployment
hetzner - Hetzner fleet configuration, updated in place
- api_key - Hetzner Cloud API key with read/write access

Read-only attributes:

id - Fleet identifier
ready - Whether the fleet configuration is healthy
status_message - Human-readable reason the fleet is not ready (present only when ready is false)
created_at - Creation timestamp
updated_at - Last update timestamp

Provider configuration, scaling_profile, and constraints are applied with full-overwrite semantics: omitting them on update resets them to their defaults. name and cluster_id are immutable, so changing either replaces the fleet.

Example:

terraform

# AWS fleet with a scaling profile and scheduling constraints
resource "cloudfleet_cfke_fleet" "aws_fleet" {
  name       = "aws-production"
  cluster_id = cloudfleet_cfke_cluster.example.id

  # aggressive (cost-optimized) or conservative (stability)
  scaling_profile = "conservative"

  limits {
    cpu = 50.0
  }

  aws {
    role_arn = "arn:aws:iam::123456789012:role/CloudfleetKarpenter"
  }

  # Restrict which nodes the fleet may provision. Omit a set for no restriction.
  constraints {
    arch          = ["amd64", "arm64"]
    capacity_type = ["on-demand", "spot"]
  }
}

# Multi-cloud fleet
resource "cloudfleet_cfke_fleet" "multi_cloud" {
  name       = "multi-cloud-production"
  cluster_id = cloudfleet_cfke_cluster.example.id

  limits {
    cpu = 100.0
  }

  aws {
    role_arn = "arn:aws:iam::123456789012:role/CloudfleetKarpenter"
  }

  gcp {
    project_id = "my-gcp-project-123"
  }

  hetzner {
    api_key = var.hetzner_api_key
  }
}

cloudfleet_cfke_node_join_information

This resource generates cloud-init userdata for joining self-managed nodes to CFKE clusters. It enables you to provision nodes across any cloud provider or on-premises infrastructure, extending Cloudfleet beyond the node auto-provisioner’s supported platforms.

On first boot, the machine configures itself and joins the cluster without any further action. The cloud-init output targets Ubuntu 22.04 and 24.04 and Debian 12 and 13 images; for nodes in the RHEL family, use cloudfleet_cfke_self_managed_node over SSH.

Required arguments:

cluster_id - Cloudfleet cluster ID to generate userdata for
zone - Availability zone for topology labels
region - Region for topology labels

Optional arguments:

base64_encode - Encode userdata with base64 (defaults to false)
gzip - Compress userdata with gzip (defaults to false)
install_nvidia_drivers - Install NVIDIA drivers on the node (defaults to false)
node_labels - Additional Kubernetes node labels (map of strings)
node_taints - Taints to register the node with (list of objects). Applied when the node first registers, like kubectl taint at registration time.
- key - Taint key (required)
- effect - Taint effect: NoSchedule, PreferNoSchedule, or NoExecute (required)
- value - Taint value (optional)

Read-only attributes:

id - Resource identifier
rendered - Generated cloud-init userdata
join_info_hash - Hash of the join information

Basic example:

terraform

# Basic node join information
resource "cloudfleet_cfke_node_join_information" "basic" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  zone       = "us-west-2a"
  region     = "us-west-2"
}

# GPU node with NVIDIA drivers and custom labels
resource "cloudfleet_cfke_node_join_information" "gpu_node" {
  cluster_id             = cloudfleet_cfke_cluster.example.id
  zone                   = "us-west-2a"
  region                 = "us-west-2"
  install_nvidia_drivers = true

  node_labels = {
    "node-type"     = "gpu-worker"
    "environment"   = "production"
    "workload-type" = "machine-learning"
  }
}

# Node registered with a taint, so only tolerating workloads are scheduled on it
resource "cloudfleet_cfke_node_join_information" "tainted" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  zone       = "us-west-2a"
  region     = "us-west-2"

  node_taints = [
    {
      key    = "dedicated"
      value  = "batch"
      effect = "NoSchedule"
    }
  ]
}

Multi-cloud provisioning examples

The cloudfleet_cfke_node_join_information resource works with any platform that supports cloud-init:

Hetzner Cloud

terraform

resource "cloudfleet_cfke_node_join_information" "hetzner" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "nbg1"
  zone       = "nbg1-dc3"

  node_labels = {
    "cfke.io/provider" = "hetzner"
  }
}

resource "hcloud_server" "worker" {
  name        = "cfke-worker-hetzner"
  image       = "ubuntu-24.04"
  server_type = "cx22"
  datacenter  = "nbg1-dc3"
  user_data   = cloudfleet_cfke_node_join_information.hetzner.rendered

  public_net {
    ipv4_enabled = true
    ipv6_enabled = true
  }
}

Scaleway

terraform

resource "cloudfleet_cfke_node_join_information" "scaleway" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "fr-par"
  zone       = "fr-par-1"

  node_labels = {
    "cfke.io/provider" = "scaleway"
  }

  # Scaleway requires uncompressed userdata
  base64_encode = false
  gzip          = false
}

resource "scaleway_instance_server" "worker" {
  name  = "cfke-worker-scaleway"
  type  = "DEV1-M"
  image = "ubuntu_jammy"

  user_data = {
    cloud-init = cloudfleet_cfke_node_join_information.scaleway.rendered
  }
}

VMware vSphere

terraform

resource "cloudfleet_cfke_node_join_information" "vmware" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "datacenter-1"
  zone       = "rack-a"

  node_labels = {
    "cfke.io/provider"    = "vmware"
    "cfke.io/environment" = "on-premises"
  }
}

resource "vsphere_virtual_machine" "worker" {
  name             = "cfke-worker-vmware"
  resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
  datastore_id     = data.vsphere_datastore.datastore.id

  num_cpus = 4
  memory   = 8192

  network_interface {
    network_id = data.vsphere_network.network.id
  }

  disk {
    label = "disk0"
    size  = 50
  }

  clone {
    template_uuid = data.vsphere_virtual_machine.ubuntu_template.id
  }

  extra_config = {
    "guestinfo.userdata.encoding" = "gzip+base64"
    "guestinfo.userdata"          = cloudfleet_cfke_node_join_information.vmware.rendered
    "guestinfo.metadata"          = base64gzip(templatefile("metadata.tftpl", {
      instance_id = "cfke-worker-vmware"
      hostname    = "cfke-worker-vmware.local"
    }))
  }
}

Global multi-cloud deployment

terraform

# Define regions for each cloud provider
locals {
  regions = {
    aws = {
      region = "us-west-2"
      zones  = ["us-west-2a", "us-west-2b"]
    }
    gcp = {
      region = "us-central1"
      zones  = ["us-central1-a", "us-central1-b"]
    }
    hetzner = {
      region = "nbg1"
      zones  = ["nbg1-dc3"]
    }
    scaleway = {
      region = "fr-par"
      zones  = ["fr-par-1", "fr-par-2"]
    }
  }
}

# Generate join information for each provider
resource "cloudfleet_cfke_node_join_information" "multi_cloud" {
  for_each = local.regions

  cluster_id = cloudfleet_cfke_cluster.global.id
  region     = each.value.region
  zone       = each.value.zones[0]

  node_labels = {
    "cfke.io/provider"   = each.key
    "cfke.io/region"     = each.value.region
    "cfke.io/deployment" = "global-production"
  }
}

# AWS instances
resource "aws_instance" "workers" {
  count         = 3
  ami           = "ami-0c2b8ca1dad447f8a"
  instance_type = "m5.large"
  user_data     = cloudfleet_cfke_node_join_information.multi_cloud["aws"].rendered

  tags = {
    Name = "cfke-worker-aws-${count.index + 1}"
  }
}

# GCP instances
resource "google_compute_instance" "workers" {
  count        = 3
  name         = "cfke-worker-gcp-${count.index + 1}"
  machine_type = "e2-standard-4"
  zone         = local.regions.gcp.zones[0]

  boot_disk {
    initialize_params {
      image = "ubuntu-os-cloud/ubuntu-2404-lts"
    }
  }

  network_interface {
    network = "default"
    access_config {}
  }

  metadata = {
    user-data = cloudfleet_cfke_node_join_information.multi_cloud["gcp"].rendered
  }
}

# Hetzner instances
resource "hcloud_server" "workers" {
  count       = 2
  name        = "cfke-worker-hetzner-${count.index + 1}"
  image       = "ubuntu-24.04"
  server_type = "cx22"
  datacenter  = "nbg1-dc3"
  user_data   = cloudfleet_cfke_node_join_information.multi_cloud["hetzner"].rendered
}

# Scaleway instances
resource "scaleway_instance_server" "workers" {
  count = 2
  name  = "cfke-worker-scaleway-${count.index + 1}"
  type  = "DEV1-M"
  image = "ubuntu_jammy"

  user_data = {
    cloud-init = cloudfleet_cfke_node_join_information.multi_cloud["scaleway"].rendered
  }
}

Workload targeting

Target specific clouds or hardware types for your workloads:

terraform

# Deploy to AWS nodes only
resource "kubernetes_deployment" "aws_workload" {
  metadata {
    name      = "aws-specific-app"
    namespace = "production"
  }

  spec {
    replicas = 3

    selector {
      match_labels = {
        app = "aws-app"
      }
    }

    template {
      metadata {
        labels = {
          app = "aws-app"
        }
      }

      spec {
        node_selector = {
          "cfke.io/provider" = "aws"
        }

        container {
          name  = "app"
          image = "nginx:latest"
        }
      }
    }
  }
}

# Deploy to GPU nodes across all clouds
resource "kubernetes_deployment" "ml_workload" {
  metadata {
    name      = "ml-training"
    namespace = "ml"
  }

  spec {
    replicas = 2

    selector {
      match_labels = {
        app = "ml-training"
      }
    }

    template {
      metadata {
        labels = {
          app = "ml-training"
        }
      }

      spec {
        node_selector = {
          "cfke.io/accelerator-manufacturer" = "NVIDIA"
        }

        container {
          name  = "trainer"
          image = "tensorflow/tensorflow:latest-gpu"

          resources {
            limits = {
              "nvidia.com/gpu" = "1"
            }
          }
        }
      }
    }
  }
}

This approach provides flexibility to run Kubernetes workloads across any infrastructure platform while maintaining centralized management through Cloudfleet.

cloudfleet_cfke_self_managed_node

Provisions Cloudfleet Kubernetes Engine (CFKE) self-managed nodes via SSH. This resource connects directly to existing infrastructure and configures it as a Kubernetes node, enabling you to integrate physical servers, existing virtual machines, or any SSH-accessible infrastructure into your CFKE cluster.

The resource configures the machine over SSH and registers it with the cluster. The machine must run Ubuntu 22.04 or 24.04, Debian 12 or 13, or the RHEL family (RHEL, Rocky Linux, AlmaLinux, CentOS Stream) 9 or 10; other releases are refused before anything is changed.

Required arguments:

cluster_id - Cloudfleet cluster ID to provision node for
region - Region for the node (added as topology labels)
zone - Availability zone for the node (added as topology labels)

Optional arguments:

install_nvidia_drivers - Install NVIDIA drivers on the node (defaults to false)
node_labels - Additional labels to apply to the Kubernetes node (merged with topology labels)
node_taints - Taints to register the node with (list of objects). Applied when the node first registers, like kubectl taint at registration time.
- key - Taint key (required)
- effect - Taint effect: NoSchedule, PreferNoSchedule, or NoExecute (required)
- value - Taint value (optional)
ssh - SSH connection configuration block

SSH configuration block:

host - SSH host address (required)
user - SSH username (required)
password - SSH password for authentication (optional, sensitive)
port - SSH port (optional, defaults to 22)
private_key_path - Path to SSH private key file for authentication (optional, sensitive)

Read-only attributes:

id - Resource identifier
join_info_hash - Hash of the join information

Example:

terraform

# Basic self-managed node with SSH key authentication
resource "cloudfleet_cfke_self_managed_node" "bare_metal" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "datacenter-1"
  zone       = "rack-a"

  ssh {
    host             = "192.168.1.100"
    user             = "ubuntu"
    private_key_path = "~/.ssh/id_rsa"
  }
}

# GPU node with NVIDIA drivers and custom labels
resource "cloudfleet_cfke_self_managed_node" "gpu_server" {
  cluster_id             = cloudfleet_cfke_cluster.example.id
  region                 = "datacenter-1"
  zone                   = "rack-b"
  install_nvidia_drivers = true

  node_labels = {
    "hardware-type"  = "gpu-server"
    "gpu-model"      = "rtx-4090"
    "environment"    = "production"
  }

  ssh {
    host             = "192.168.1.101"
    user             = "admin"
    private_key_path = "/secure/keys/gpu-server-key"
    port             = 2222
  }
}

# On-premises server with password authentication
resource "cloudfleet_cfke_self_managed_node" "legacy_server" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "on-premises"
  zone       = "legacy-datacenter"

  node_labels = {
    "server-type" = "legacy-hardware"
    "managed-by"  = "terraform"
  }

  ssh {
    host     = "legacy-server.internal"
    user     = "root"
    password = var.legacy_server_password
  }
}

# Rocky Linux server registered with a taint
resource "cloudfleet_cfke_self_managed_node" "database_host" {
  cluster_id = cloudfleet_cfke_cluster.example.id
  region     = "datacenter-1"
  zone       = "rack-c"

  node_labels = {
    "workload" = "database"
  }

  node_taints = [
    {
      key    = "dedicated"
      value  = "database"
      effect = "NoSchedule"
    }
  ]

  ssh {
    host             = "192.168.1.102"
    user             = "rocky"
    private_key_path = "~/.ssh/id_rsa"
  }
}

Previous
← Introduction

Next
Data sources →