Framework Documentation

GitOps Process

TL;DR:

• Kubestack's GitOps process is designed to make infrastructure automation reliable.
• Kubestack differentiates between infrastructure and application environments.
• Any change follows the same review, validate, and promote workflow.
• Git branches and tags determine the environment and pipeline steps.

GitOps prerequisites

Adopting GitOps requires reliable infrastructure automation. Kubestack combines an intuitive change management process with at least one internal and one external environment to achieve this.

The change management process uses Git branches and tags to trigger the pipeline. The pipeline will either provide feedback for peer-reviews, validate changes against the internal environment or promote the change to the external environment.

This enables teams to jointly maintain the configuration of Kubernetes infrastructure and Kubernetes services in a reliable and automated way.

Infrastructure environments

The Kubestack framework uses Terraform workspaces to provide independent infrastructure environments. By default, there are either two or three environments ops, apps or ops, apps and apps-prod depending on your infrastructure environment choice. But the number of environments and their names can be changed.

• ops: validates changes against real cloud infrastructure before they get promoted to the mission critical environment
• apps: provides the stable environment for mission critical applications (home to either non-prod and prod or only non-prod namespaces)
• (optional) apps-prod: provides the stable environment for mission critical applications (home to prod namespaces only)

Keeping infrastructure and application environments independent is key for reliable automation. Clear separation of concerns also allows teams to work on infrastructure and applications simultaneously.

Calling the environments ops, apps and apps-prod aims to make their purpose clear and avoid misunderstandings.

Making changes

The Kubestack workflow is very similar to GitHub's Git-flow. Changing and applying configuration is a four-step process. Each Git push triggers a pipeline run. The pipeline will run Terraform, which will send the new committed state to both the cloud provider or Kubernetess API. Synchronizing the new desired state with the current state continuously is the responsibility of the active cloud and Kubernetes control planes.

1. Change

   Make changes to the configuration in a new branch. Commit the changed configuration. Validate your changes by pushing the new branch. The pipeline runs a Terraform plan against the ops environment.

2. Review

   Request a peer review of your changes. Team members review the changes and the Terraform plan. If reviewers require changes, make additional commits in the branch.

3. Validate

   If approved, merge your changes to the main branch, to apply them against the ops environment. After applying to ops was successful, the pipeline runs a Terraform plan against the apps environment.

4. Promote

   Review the previous apps environment plan and tag the merge commit to promote the same changes to the apps environment.

Pipeline triggers and steps

Any commit pushed to a remote repository should trigger a pipeline run. The branch or tag of the commit that triggered the run determines the correct environment and steps. The pipeline selects the workspace for the environment and runs the required steps.

Kubestack knows three types of triggers:

1. Triggers from feature branches.

   Feature branch pipeline-runs provide a Terraform plan for the ops workspace as feedback for reviewers. Teams should peer review the changes and the plan before merging the feature branch into the main branch.

   The pipeline runs the following Terraform commands for feature branches:

   1. terraform init
   2. terraform workspace select ops
   3. terraform plan

2. Triggers from the main branch.

   Main branch pipeline-runs validate changes by applying them against the ops workspace. Terraform apply can fail even if the prior plan was successful. Applying changes to the dedicated ops environment reduces the risk of this happening when applying to the critical apps environment.

   The pipeline runs the following Terraform commands for the main branch:

   1. terraform init
   2. terraform workspace select ops
   3. terraform plan
   4. terraform apply
   5. terraform workspace select apps
   6. terraform plan

3. Triggers from deploy-tags.

   Deploy-tag pipeline-runs apply the configuration against the apps workspace. Tags point to a specific commit in the Git repository. By using tags the exact same configuration is promoted from the ops to the apps environment.

   The pipeline runs the following Terraform commands for deploy-tags:

   1. terraform init
   2. terraform workspace select apps
   3. terraform plan
   4. terraform apply