Amazon EBS

The Amazon EBS CSI driver provisions EBS volumes for Pods on the AWS nodes of your Fleet. The driver authenticates with an IAM role that trusts your cluster’s OIDC tokens. You store no AWS access keys in the cluster.

Prerequisites

  • A CFKE cluster with an AWS Fleet
  • Permission to create IAM roles and OIDC identity providers in the AWS account
  • Helm and kubectl configured for the cluster

The examples use the cluster ID CLUSTER_ID. Replace it with the ID of your cluster.

Step 1: Create the IAM role

The EBS CSI controller runs as the ebs-csi-controller-sa service account in the kube-system namespace. Create an IAM role that this service account can assume with its OIDC token, and attach the AWS-managed AmazonEBSCSIDriverPolicy to it.

The following Terraform configuration creates the OIDC identity provider for the cluster and the role. If you already created an OIDC identity provider for the cluster, for example to access cloud APIs, reference it instead of creating a second one.

terraform 
locals {
  cluster_id = "CLUSTER_ID"
  issuer     = "https://api.cloudfleet.ai/v1/clusters/${local.cluster_id}"
}

resource "aws_iam_openid_connect_provider" "cfke" {
  url            = local.issuer
  client_id_list = [local.issuer]
}

data "aws_iam_policy_document" "ebs_csi_trust" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.cfke.arn]
    }
    condition {
      test     = "StringEquals"
      variable = "api.cloudfleet.ai/v1/clusters/${local.cluster_id}:aud"
      values   = [local.issuer]
    }
    condition {
      test     = "StringEquals"
      variable = "api.cloudfleet.ai/v1/clusters/${local.cluster_id}:sub"
      values   = ["system:serviceaccount:kube-system:ebs-csi-controller-sa"]
    }
  }
}

resource "aws_iam_role" "ebs_csi" {
  name               = "cfke-ebs-csi-${local.cluster_id}"
  assume_role_policy = data.aws_iam_policy_document.ebs_csi_trust.json
}

resource "aws_iam_role_policy_attachment" "ebs_csi" {
  role       = aws_iam_role.ebs_csi.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

output "ebs_csi_role_arn" {
  value = aws_iam_role.ebs_csi.arn
}

The trust policy accepts only tokens of the controller’s service account in your cluster. To create the same resources with the AWS CLI, follow the AWS example in accessing cloud APIs securely.

Step 2: Install the driver

Create a file named ebs-csi.yaml. Replace ROLE_ARN with the ARN of the role from step 1:

yaml 
controller:
  nodeSelector:
    cfke.io/provider: aws
  env:
    - name: AWS_ROLE_ARN
      value: ROLE_ARN
    - name: AWS_WEB_IDENTITY_TOKEN_FILE
      value: /var/run/secrets/kubernetes.io/serviceaccount/token
node:
  nodeSelector:
    cfke.io/provider: aws

The controller exchanges the Pod’s service account token for credentials of the IAM role. The controller and the node plugin run on AWS nodes.

Install the driver with Helm:

bash 
helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
helm repo update aws-ebs-csi-driver
helm upgrade --install aws-ebs-csi-driver aws-ebs-csi-driver/aws-ebs-csi-driver \
  -n kube-system --values ebs-csi.yaml

Check that the controller and the node plugins are running:

bash 
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver

The cluster runs two controller Pods and one node plugin Pod on each AWS node. If the cluster has no AWS node yet, the node auto-provisioner adds one for the controller.

Step 3: Create a storage class

Create a storage class for EBS volumes:

yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
  type: gp3

For other volume types or encryption, see the storage class parameters.

Step 4: Use a volume

Create a StatefulSet that requests an EBS volume and runs on AWS nodes:

yaml 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ebs-example
spec:
  serviceName: ebs-example
  replicas: 1
  selector:
    matchLabels:
      app: ebs-example
  template:
    metadata:
      labels:
        app: ebs-example
    spec:
      nodeSelector:
        cfke.io/provider: aws
      containers:
        - name: app
          image: busybox:1.37
          command: ["sh", "-c", "date >> /data/boots; cat /data/boots; exec sleep infinity"]
          resources:
            requests:
              cpu: 10m
              memory: 16Mi
          volumeMounts:
            - name: data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: ebs
        resources:
          requests:
            storage: 10Gi

Check that the claim is bound:

bash 
kubectl get pvc data-ebs-example-0

Each start of the Pod appends a line to /data/boots. Delete the Pod, or the node it runs on, and check the log of the new Pod. It lists the earlier starts, because the volume moved with the Pod to a node in the same availability zone.

Next steps

Previous
← Storage overview
Next
GCP Persistent Disk →
On this page