Clusters

TL;DR:

Clusters are added by calling cluster modules.
These modules provision managed Kubernetes clusters and their direct cloud infrastructure dependencies.
A platform can have one or more clusters and mix multiple regions and even different cloud providers.

Adding or Removing Clusters

Clusters can be added or removed by adding or removing cluster modules. The easiest way to do so is to ask your AI coding agent that has learned the Kubestack skill to scaffold the module for you.

Following the skill, your agent only makes changes to local files, and you can see the changes with git status. To have the changes take effect, you have to commit, push, merge and promote them following the GitOps flow.

Adding a Cluster

Adding a cluster adds two files to your repository. The first one, calls the cluster module and the second one configures the required providers. Ask your agent to scaffold the cluster for you. Your agent configures the base domain and environments of the new cluster to match your platform automatically. You can of course also write the configuration yourself, the configuration section below documents all available attributes.

AmazonAzureGoogleScaleway

Adding an EKS cluster requires specifying the name prefix and the region.

Add an EKS cluster with name_prefix gc0 in the eu-west-1 region

Removing a Cluster

You can also ask your agent to remove a cluster for you. Removing a cluster will also remove all the cluster's node pools and services. Your agent follows the skill's staged removal process and prepares the changes one stage at a time.

# list all platform components
git ls-files '*_cluster.tf'
aks_gc0_westeurope_cluster.tf
eks_gc0_eu-west-1_cluster.tf
gke_gc0_europe-west1_cluster.tf

# then ask your agent to remove the desired cluster by <name>
Remove the eks_gc0_eu-west-1 cluster

Configuration

Cluster module configuration is provider specific. See the commenteded examples below for available attributes.

AmazonAzureGoogleScaleway

module "eks_gc0_eu-west-1" {
  providers = {
    aws = aws.eks_gc0_eu-west-1
  }

  source = "github.com/kbst/terraform-kubestack//aws/cluster?ref=<version>"

  configuration = {
    apps = {
      # prefix added to workspace and region for the cluster name (required)
      name_prefix = "gc0"

      # part of the fully qualified domain name (required)
      base_domain = var.base_domain

      # list of availability zones to use, requires at least two zones
      # must match the region configured on the provider (required)
      cluster_availability_zones = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]

      # configuration of the default node-pool (required)
      default_node_pool = {
        # list of EC2 instance types to use for the default node-pool (required)
        instance_types = ["t3a.xlarge"]

        # min and max number of nodes (required)
        min_size = 3
        max_size = 9

        # desired number of nodes
        # (optional), defaults to 1
        desired_capacity = 3

        # size in GB for root device volumes of worker nodes
        # (optional), defaults to 20
        root_device_volume_size = 20

        # instance metadata options for the default node-pool's nodes
        # (optional), defaults to null
        metadata_options = {
          http_endpoint               = "enabled"
          http_tokens                 = "required"
          http_put_response_hop_limit = 1
          http_protocol_ipv6          = "disabled"
          instance_metadata_tags      = "disabled"
        }

        # AMI type to use for nodes
        # (optional), default depends on instance types
        # AL2023_x86_64_STANDARD, AL2023_ARM_64_STANDARD or AL2023_x86_64_NVIDIA
        ami_type = "AL2023_x86_64_STANDARD"
      }

      # desired Kubernetes version, set to upgrade, downgrades are not supported by EKS
      # (optional), defaults to latest at creation time
      cluster_version = "1.30"

      # sets `mapAccounts` attribute in the `aws-auth` configmap in the `kube-system` namespace
      # (optional), defaults to null
      cluster_aws_auth_map_accounts = <<-MAPACCOUNTS
      - "000000000000"
      MAPACCOUNTS

      # sets `mapRoles` attribute in the `aws-auth` configmap in the `kube-system` namespace
      # (optional), defaults to null
      cluster_aws_auth_map_roles = <<-MAPROLES
      - rolearn: arn:aws:iam::000000000000:role/KubernetesAdmin
        username: kubernetes-admin
        groups:
        - system:masters
      MAPROLES

      # sets `mapUsers` attribute in the `aws-auth` configmap in the `kube-system` namespace
      # (optional), defaults to null
      cluster_aws_auth_map_users = <<-MAPUSERS
      - userarn: arn:aws:iam::000000000000:user/Alice
        username: alice
        groups:
        - system:masters
      MAPUSERS

      # list of control-plane log types to enable
      # (optional), defaults to ["api", "audit", "authenticator", "controllerManager", "scheduler"]
      enabled_cluster_log_types = ["audit", "authenticator"]
    }

    ops = {}
  }
}