Framework Documentation

TL;DR:

• Kubestack's GitOps process is designed to make infrastructure automation reliable.
• Kubestack differentiates between infrastructure and application environments.
• Any change follows the same review, validate, and promote workflow.
• Git branches and tags determine the environment and pipeline steps.

Adopting GitOps requires reliable infrastructure automation. Kubestack combines an intuitive change management process with at least two dedicated environments, by default called ops and apps, to achieve this.

The change management process uses Git branches and tags to trigger the pipeline. The pipeline will either provide feedback for peer-reviews, validate changes against the ops environment or promote the change to the apps environment.

This enables teams to jointly maintain the configuration of Kubernetes infrastructure and Kubernetes services in a reliable and automated way.

The Kubestack framework uses Terraform workspaces to provide independent infrastructure environments. By default, there are either two or three environments ops, apps or ops, apps and apps-prod depending on your infrastructure environment choice. But the number of environments and their names can be changed.

• ops: validates changes against real cloud infrastructure before they get promoted to the mission critical environment
• apps: provides the stable environment for mission critical applications (home to either non-prod and prod or only non-prod namespaces)
• (optional) apps-prod: provides the stable environment for mission critical applications (home to prod namespaces only)

These infrastructure environments should not be confused with application environments. Kubestack explicitly avoids common application environment names like dev, staging or prod for its infrastructure environments.

Keeping infrastructure and application environments independent is key for reliable automation. Clear separation of concerns also allows teams to work on infrastructure and applications simultaneously.

Calling the environments ops, apps and apps-prod aims to make their purpose clear and avoid misunderstandings.

Custom environments

The default environments can be changed. It is possible to configure custom names and a different number of environments. This allows alternative cluster-environment-architectures like for example ops, apps-stage and apps-prod for teams that require staging and production application environments to be on different clusters.

Custom environments require modifications to the following areas:

1. Create Terraform workspaces that match your custom environment names
2. Adapt the configuration hash keys for all modules to match the names of your custom environments
3. Set the configuration_base_key for all modules to the name of the custom environment that all other environments should inherit from
4. Modify the CI/CD pipeline so that it provides feedback, validates or promotes changes according to your custom environments

The Kubestack documentation is using the ops and apps environments by default.

The Kubestack workflow is very similar to GitHub's Git-flow. Changing and applying configuration is a four-step process:

1. Change

   Make changes to the configuration in a new branch. Commit the changed configuration. Validate your changes by pushing the new branch. The pipeline runs a Terraform plan against the ops environment.

2. Review

   Request a peer review of your changes. Team members review the changes and the Terraform plan. If reviewers require changes, make additional commits in the branch.

3. Validate

   If approved, merge your changes to the main branch, to apply them against the ops environment. After applying to ops was successful, the pipeline runs a Terraform plan against the apps environment.

4. Promote

   Review the previous apps environment plan and tag the merge commit to promote the same changes to the apps environment.

Any commit pushed to a remote repository should trigger a pipeline run. The branch or tag of the commit that triggered the run determines the correct environment and steps. The pipeline selects the workspace for the environment and runs the required steps.

Kubestack knows three types of triggers:

1. Triggers from feature branches.

   Feature branch pipeline-runs provide a Terraform plan for the ops workspace as feedback for reviewers. Teams should peer review the changes and the plan before merging the feature branch into the main branch.

   The pipeline runs the following Terraform commands for feature branches:

   1. terraform init
   2. terraform workspace select ops
   3. terraform plan

2. Triggers from the main branch.

   Main branch pipeline-runs validate changes by applying them against the ops workspace. Terraform apply can fail even if the prior plan was successful. Applying changes to the dedicated ops environment reduces the risk of this happening when applying to the critical apps environment.

   The pipeline runs the following Terraform commands for the main branch:

   1. terraform init
   2. terraform workspace select ops
   3. terraform plan
   4. terraform apply
   5. terraform workspace select apps
   6. terraform plan

3. Triggers from deploy-tags.

   Deploy-tag pipeline-runs apply the configuration against the apps workspace. Tags point to a specific commit in the Git repository. By using tags the exact same configuration is promoted from the ops to the apps environment.

   The pipeline runs the following Terraform commands for deploy-tags:

   1. terraform init
   2. terraform workspace select apps
   3. terraform plan
   4. terraform apply