Set up Automation
Overview
To set up the automation we need to:
- Push your repository.
- Set up pipeline credentials.
- Add the pipeline file.
- Follow the GitOps process.
Github is used for the step-by-step instructions. But you can use other Git hosting and CI/CD providers too. The required steps tend to be similar.
Example pipelines for other CI/CD systems are available on Github.
Push your repository
Create the remote repository
Create a new repository and give it a descriptive name. For example
infrastructure-automation
.Push the repository
Follow option two from Github's instructions to push your existing repository.
Set up pipeline credentials
Encode the AWS credentials file using base64.
cat .user/.aws/credentials | base64 -w 0 && echoFind secrets under your repository settings and add a secret named
KBST_AUTH_AWS
with the base64 encoded credentials from the previous step as the value.
Encode the service principal credentials using base64
cat .user/.azure/KBST_AUTH_AZ | base64 -w 0 && echoFind secrets under your repository settings and add a secret named
KBST_AUTH_AZ
with the base64 encoded credentials from the previous step as the value.
Encode the service account key using base64
cat .user/.config/gcloud/application_default_credentials.json | base64 -w 0 && echoFind secrets under your repository settings and add a secret named
KBST_AUTH_GCLOUD
with the base64 encoded credentials from the previous step as the value.
Add the pipeline file
First create a new branch to work in.
git checkout -b ghactionsAdd the pipeline as
.github/workflows/main.yaml
.mkdir -p .github/workflowscat > .github/workflows/main.yaml <<'EOF'name: deployon:push:branches:- "**" # run for branchestags:- "*" # run for tagsjobs:deploy:runs-on: ubuntu-latestenv:KBST_DOCKER_ARGS: --rm -v ${{ github.workspace }}:/infra -e AWS_EC2_METADATA_DISABLED=true -e TF_IN_AUTOMATION=trueKBST_DOCKER_IMAGE: kbst:${{ github.sha }}steps:- uses: actions/checkout@v1### Build image- name: Build imageenv:DOCKER_BUILDKIT: 1run: docker build -t $KBST_DOCKER_IMAGE .### Terraform init- name: Terraform initenv:KBST_AUTH_AWS: ${{ secrets.KBST_AUTH_AWS }}KBST_AUTH_AZ: ${{ secrets.KBST_AUTH_AZ }}KBST_AUTH_GCLOUD: ${{ secrets.KBST_AUTH_GCLOUD }}run: |docker run \$KBST_DOCKER_ARGS \-e KBST_AUTH_AWS \-e KBST_AUTH_AZ \-e KBST_AUTH_GCLOUD \$KBST_DOCKER_IMAGE \terraform init### Select ops or apps workspace- name: Select ops workspaceif: startsWith(github.ref, 'refs/tags/apps-deploy-') == falserun: |docker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform workspace select ops- name: Select apps workspaceif: startsWith(github.ref, 'refs/tags/apps-deploy-')run: |docker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform workspace select apps### Terraform plan against current workspace- name: Terraform planrun: |docker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform plan --out=tfplan --input=false### Terraform apply against current workspace# for main branch or apps-deploy tag- name: Terraform applyif: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/apps-deploy-')run: |docker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform apply --input=false tfplan### Terraform plan against apps workspace# after applying to ops,# run plan against apps- name: Terraform planif: github.ref == 'refs/heads/main'run: |docker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform workspace select appsdocker run \$KBST_DOCKER_ARGS \$KBST_DOCKER_IMAGE \terraform plan --out=tfplan --input=falseEOF
Follow the GitOps process
Add, commit and push the pipeline.
git add .git commit -m "Add Github Actions pipeline"git push origin ghactionsOpen a pull request.
Check the pipeline run.
The pipeline run for the
ghactions
branch does not apply changes. It only provides the output ofterraform plan
against the ops workspace to determine what changes will be applied once merged.Since we already bootstrapped the clusters, the pipeline at this point has no planned changes.
Merge the pull request to apply changes to ops.
Merge the pull request into main. The pipeline applies changes against the ops workspace on every commit to main.
Finally, set a tag to apply changes to apps.
# checkout the main branchgit checkout main# pull changes from origingit pull# tag the merge commitgit tag apps-deploy-0# push the tag to origin to trigger the pipelinegit push origin apps-deploy-0
Compare the output of the three pipeline runs. You will see how when triggered from a feature branch, triggered from the main branch or triggered from a tag the pipeline behaves differently.
For more details refer to the GitOps Flow making changes section.
Recap
To recap:
- You bootstrapped a local repository.
- Created prerequisites like the Terraform remote state and the identity for the automation runs.
- You provisioned both the ops and apps infrastructure environments.
- Setup DNS and Ingress to be able to expose workloads.
- You linked your repository to trigger automated pipeline runs.
Congratulations, you now have a fully automated GitOps infrastructure.
Professional Services: For organizations interested in accelerating their GitOps journey.
- POC Workshops
- Enterprise Integration
- Training and Support