Framework Guides

TL;DR:

• Kubestack catalog modules can be used standalone, without using the whole Kubestack framework
• This requires setting configuration_base_key and configuration attributes to match your Terraform workspace(s)
• The implicitly or explicitly injected Kustomization provider controls the cluster the resources are applied to

Kubestack catalog modules use the Kubestack maintained Kustomization Terraform provider and package Kubernetes YAML from upstream releases of popular Kubernetes cluster services. The modules are built using a pipeline that checks for new upstream releases daily and then packages and tests the new upstream release as a Terraform module. If the tests pass, the new version is made available on the Kubestack module registry.

The modules, due to using the Kustomization provider, have two main benefits over comparable helm or kubectl based Terraform modules:

1. Manifests are fully customiseable using Kustomize attributes set as part of the module call. This makes maintaining typical customisations low effort across future version updates.
2. All upstream Kubernetes resources are fully integrated into the Terraform state and therefor Terraform plan/apply lifecycle. This means you can see changes to individual Kubernetes manifests during plan and resources deleted from manifests are reliably purged from the clusters. Using server side dry-runs additionally ensures that changes to immutable Kubernetes fields produce a destroy-and-recreate plan for the respective resource.

For a detailed comparison between a Helm provider and a Kustomization provider based module take a look at the blog post: A better way to provision Kubernetes resources using Terraform

1. Call the module

   The modules are called like any other Terraform module. You specify the source and version attributes to control the upstream service and version to apply on your cluster. The version attribute has the upstream release version, and a Kubestack packaging suffix. E.g. 1.0.0-kbst.0. The kbst.0 is increased if there are changes to the module, without the upstream release version changing.

   module "example_catalog_module" {
     # [...]
   
     source  = "kbst.xyz/catalog/CATALOG_ENTRY_NAME/kustomization"
     version = "CATALOG_ENTRY_VERSION"
   
     # [...]
   }
   
   

2. Inject a configured Kustomization provider

   To control what cluster the module's Kubernetes resources are applied to, the provider needs to be configured with a valid kubeconfig. If you only have a single cluster and respectively configured provider, you can use the provider that Terraform passes into the module implicitly. But if you have mulitple clusters, create one aliased Kustomization provider per cluster. And explicitly inject the correct aliased provider into the catalog module.

   module "example_catalog_module" {
     providers = {
       kustomization = kustomization.YOUR_ALIAS_NAME
     }
   
     # [...]
   }
   
   

3. Set configuration_base_key and configuration to match your Terraform workspace(s)

   Kubestack uses Terraform workspaces to implement its infrastructure environments. It also has configuration inheritance from the mission critical to the non mission critical environments. The cluster service and custom manifest modules from the catalog fully integrate with these framework features.

   But what workspace/environment to inherit from and how many environments there are can be configured. The configuration_base_key attribute sets which workspace name to inherit from. And the keys in the configuration hash map define the names and number of environments.

   module "example_catalog_module" {
     # [...]
   
     configuration_base_key = "default"
     configuration = {
       default = {}
     }
   }
   
   

   You can list the Terraform workspaces in your state using terraform workspace list. There is always at least the default workspace.

   $ terraform workspace list
   * default
   
   

Kustomization Attributes

Modules allow customisation of the upstream manifests. This is achieved by integrating Kustomize into Terraform using the Kustomization provider. The modules allow setting the Kustomize attributes as part of the module's configuration attribute per workspace.

For this reason, configuration is a hash map, where the key is the workspace name, e.g. default and the value is a hash map of Kustomize attributes.

Full documentation of the configuration attribute can be found in the cluster service modules documentation.

Example for Nginx ingress module

This example shows how to call the Nginx ingress module without the Kubestack framework. It also scales the ingress controller replicas to 5, as an example of customizing the values from upstream in the module call.

Require and configure provider

terraform {
  required_providers {
    kustomization = {
      source = "kbst/kustomization"
    }
  }
}

provider "kustomization" {
  alias = "local"

  kubeconfig_path = "~/.kube/config"
}

Call module

module "example_nginx" {
  providers = {
    # we're using the alias provider we configured above
    kustomization = kustomization.local
  }

  source  = "kbst.xyz/catalog/nginx/kustomization"
  version = "1.2.1-kbst.0" # find the latest version on https://www.kubestack.com/catalog/nginx

  # the configuration here assumes you're using Terraform's default workspace
  # use `terraform workspace list` to see the workspaces
  configuration_base_key = "default"
  configuration = {
    default = {
      replicas = [{
        name  = "ingress-nginx-controller"
        count = 5
      }]
    }
  }
}