Repository layout

TL;DR:

  • Kubestack follows a convention over configuration approach.
  • The directory layout follows common Terraform recommendations.
  • For multi-cluster support, cluster specific files are prefixed with the cluster module name.

Repository root

Terraform evaluates all .tf files in the same directory into a single configuration. It is recommended to split up the infrastructure configuration into different files for readability. Kubestack aims to follow popular Terraform conventions. So teams familiar with Terraform should feel right at home.

Files and directories in the root of the Kubestack repository fall into two categories:

  1. Common files and directories that are not specific to a cluster module or cluster service module
  2. Per cluster files that are prefixed with the respective cluster module name

Common files and directories

  • .user/

    Kubestack provides container images to be used during bootstrapping and for disaster recovery locally. The .user directory is used to persist the HOME directory between containers. It is excluded from Git using the .gitignore file in the repository.

  • manifests/

    The manifests directory can be used to store custom Kubernetes manifests in YAML format that may be integrated using the custom-manifest module.

  • Dockerfile and Dockerfile.loc

    The two Dockerfiles specify the version of the Kubestack container image to use. You can use the Dockerfiles to add additional binaries to Kubestack images. Or customize the environment and configuration of the upstream image. The Dockerfile.loc file is used for the local development environment started with kbst local apply.

  • state.tf

    This file defines the configuration for the Terraform remote state. Terraform stores state about your infrastructure. It uses this state to map the real world resources to your configuration, keep track of metadata and to improve performance.

    When using Terraform as a team and in a CI/CD pipeline it is important to store state centrally and ensure only one change is applied at any given time. This is achieved by using remote state with state locking.

    Remote state backends are not Kubestack specific. You can use any backend Terraform supports. Please refer to the Terraform documentation for backends. The tutorial defaults to using your chosen cloud provider's object storage as the Terraform remote state store.

  • versions.tf and .terraform.lock.hcl

    The versions file is used to set the required Terraform version and specify required Terraform providers. A Terraform init will determine module and provider versions to use, based on the required versions from the root and all child modules. The resulting provider versions are locked using the Terraform lock file. This ensures subsequent Terraform runs use the same provider versions.

Per cluster files

Kubestack supports managing multiple clusters, even in different regions and from different cloud providers, in a single repository. In addition, Kubestack supports provisioning cluster services on top of these clusters. This means there will be one cluster module and multiple cluster service modules per desired cluster. To organize this, Kubestack recommends a convention to prefix these cluster specific Terraform files with the module name.

The following files prefix example_ as a placeholder for the module name in a real configuration.

  • example_cluster.tf

    Use a file per desired cluster by calling a cluster module and passing the respective configuration. In a multi-cluster configuration you'd have multiple *_cluster.tf files. Modules are self-contained packages of reusable Terraform configurations.

  • example_providers.tf

    Providers extend Terraform to support additional APIs. In Kubestack's case, the main providers used are the provider for your chosen cloud. And the Kubestack maintained Kustomization provider. The cloud API credentials, are read from the environment automatically. Some cloud providers require per provider configuration. But to deploy Kubernetes resources onto the correct cluster, each cluster has a specific Kustomization provider. These provider configurations are defined in the per cluster providers file.

  • example_cluster_service.tf

    The cluster service modules, that use the cluster specific Kustomization provider configuration are also recommended to be stored in individual files.

Example layout
.
├── Dockerfile
├── Dockerfile.loc
├── .gitignore
├── example_cluster.tf
├── example_nginx.tf
├── example_prometheus.tf
├── example_providers.tf
├── example_sealed_secrets.tf
├── manifests/
├── README.md
├── .terraform.lock.hcl
├── .user/
└── versions.tf