Using the Kubestack catalog modules without the Kubestack framework
TL;DR:
- Kubestack catalog modules can be used standalone, without using the whole Kubestack framework
- This requires setting
configuration_base_key
andconfiguration
attributes to match your Terraform workspace(s) - The implicitly or explicitly injected Kustomization provider controls the cluster the resources are applied to
Introduction
Kubestack catalog modules use the Kubestack maintained Kustomization Terraform provider and package Kubernetes YAML from upstream releases of popular Kubernetes cluster services. The modules are built using a pipeline that checks for new upstream releases daily and then packages and tests the new upstream release as a Terraform module. If the tests pass, the new version is made available on the Kubestack module registry.
The modules, due to using the Kustomization provider, have two main benefits over comparable helm or kubectl based Terraform modules:
- Manifests are fully customiseable using Kustomize attributes set as part of the module call. This makes maintaining typical customisations low effort across future version updates.
- All upstream Kubernetes resources are fully integrated into the Terraform state and therefor Terraform plan/apply lifecycle. This means you can see changes to individual Kubernetes manifests during plan and resources deleted from manifests are reliably purged from the clusters. Using server side dry-runs additionally ensures that changes to immutable Kubernetes fields produce a destroy-and-recreate plan for the respective resource.
For a detailed comparison between a Helm provider and a Kustomization provider based module take a look at the blog post: A better way to provision Kubernetes resources using Terraform
Module Installation
Call the module
The modules are called like any other Terraform module. You specify the
source
andversion
attributes to control the upstream service and version to apply on your cluster. Theversion
attribute has the upstream release version, and a Kubestack packaging suffix. E.g.1.0.0-kbst.0
. Thekbst.0
is increased if there are changes to the module, without the upstream release version changing.module "example_catalog_module" {# [...]source = "kbst.xyz/catalog/CATALOG_ENTRY_NAME/kustomization"version = "CATALOG_ENTRY_VERSION"# [...]}Inject a configured Kustomization provider
To control what cluster the module's Kubernetes resources are applied to, the provider needs to be configured with a valid kubeconfig. If you only have a single cluster and respectively configured provider, you can use the provider that Terraform passes into the module implicitly. But if you have mulitple clusters, create one aliased Kustomization provider per cluster. And explicitly inject the correct aliased provider into the catalog module.
module "example_catalog_module" {providers = {kustomization = kustomization.YOUR_ALIAS_NAME}# [...]}Set
configuration_base_key
andconfiguration
to match your Terraform workspace(s)Kubestack uses Terraform workspaces to implement its infrastructure environments. It also has configuration inheritance from the mission critical to the non mission critical environments. The cluster service and custom manifest modules from the catalog fully integrate with these framework features.
But what workspace/environment to inherit from and how many environments there are can be configured. The
configuration_base_key
attribute sets which workspace name to inherit from. And the keys in theconfiguration
hash map define the names and number of environments.module "example_catalog_module" {# [...]configuration_base_key = "default"configuration = {default = {}}}You can list the Terraform workspaces in your state using
terraform workspace list
. There is always at least thedefault
workspace.$ terraform workspace list* default
Module Configuration
Kustomization Attributes
Modules allow customisation of the upstream manifests.
This is achieved by integrating Kustomize into Terraform using the Kustomization provider.
The modules allow setting the Kustomize attributes as part of the module's configuration
attribute per workspace.
For this reason, configuration
is a hash map, where the key is the workspace name, e.g. default
and the value is a hash map of Kustomize attributes.
Full documentation of the configuration
attribute can be found in the cluster service modules documentation.
Example for Nginx ingress module
This example shows how to call the Nginx ingress module without the Kubestack framework. It also scales the ingress controller replicas to 5, as an example of customizing the values from upstream in the module call.
Require and configure provider
terraform {required_providers {kustomization = {source = "kbst/kustomization"}}}provider "kustomization" {alias = "local"kubeconfig_path = "~/.kube/config"}
Call module
module "example_nginx" {providers = {# we're using the alias provider we configured abovekustomization = kustomization.local}source = "kbst.xyz/catalog/nginx/kustomization"version = "1.2.1-kbst.0" # find the latest version on https://www.kubestack.com/catalog/nginx# the configuration here assumes you're using Terraform's default workspace# use `terraform workspace list` to see the workspacesconfiguration_base_key = "default"configuration = {default = {replicas = [{name = "ingress-nginx-controller"count = 5}]}}}