Elasticsearch Operator

by  UPMC Enterprises

The Elasticsearch operator provisions and configures an Elasticsearch cluster on top of Kubernetes. Elasticsearch is a distributed, JSON-based search and analytics engine designed for horizontal scalability, maximum reliability and easy management.

In addition to Elasticsearch the operator also includes additional pieces of the Elastic stack, like Kibana.

TL;DR:

  • Installing, updating or removing follows Kubestack's GitOps flow.
  • Instructions assume the default repository layout.
  • Bases can be consumed as-is or customized.
  • Step-by-step instructions are framework specific but bases can be used independently.

Install

  1. Vendor the base

    # Run these commands from the root of your Kubestack infra repository
    wget https://storage.googleapis.com/catalog.kubestack.com/elasticsearch-v0.4.0-kbst.1.zip
    unzip -d manifests/bases/ elasticsearch-v0.4.0-kbst.1.zip
    rm elasticsearch-v0.4.0-kbst.1.zip
  2. Include resource in apps overlay

    cd manifests/overlays/apps
    kustomize edit add resource ../../bases/elasticsearch/clusterwide
  3. Commit and push

    cd -
    git checkout -b add-elasticsearch
    git add manifests/bases/elasticsearch manifests/overlays/apps/kustomization.yaml
    git commit -m "Add elasticsearch v0.4.0-kbst.1 base"
    git push origin add-elasticsearch
  4. Review PR and merge

    Finally, review and merge the PR into master. Once it's been successfully applied against the Ops-Cluster set a prod-deploy tag to also apply the change against the Apps-Cluster.

Update

To update the operator delete the previously vendored base and then vendor the new version.

  1. Delete the previous vendored version

    # Run these commands from the root of your Kubestack infra repository
    rm -r manifests/bases/elasticsearch
  2. Vendor the new version

    # Run these commands from the root of your Kubestack infra repository
    wget https://storage.googleapis.com/catalog.kubestack.com/elasticsearch-v0.4.0-kbst.1.zip
    unzip -d manifests/bases/ elasticsearch-v0.4.0-kbst.1.zip
    rm elasticsearch-v0.4.0-kbst.1.zip
  3. Commit and push

    git checkout -b update-elasticsearch
    git add manifests/bases/elasticsearch
    git commit -m "Update elasticsearch base to v0.4.0-kbst.1"
    git push origin update-elasticsearch

Remove

Operators often create resources based on custom objects. When removing an operator, follow a two-step process to ensure operator provisioned resources are purged properly.

  1. Remove all the operator's custom objects.
  2. Once the operator had time to de-provision all resources it created, follow the instructions below to remove the operator itself.
  1. Remove resource from apps overlay

    cd manifests/overlays/apps
    kustomize edit remove resource ../../bases/elasticsearch/clusterwide
  2. Delete the vendored base from your repository

    cd -
    # Run these commands from the root of your Kubestack infra repository
    rm -r manifests/bases/elasticsearch
  3. Commit and push

    git checkout -b remove-elasticsearch
    git add manifests/bases/elasticsearch
    git commit -m "Remove elasticsearch base"
    git push origin remove-elasticsearch

Usage

Once the operator has been deployed to the Kubernetes cluster, you can use it to provision and operate one or more Elasticsearch clusters by creating a custom object.

Elasticsearch Custom Object

Below is an example of a ElasticsearchCluster resource to instruct the operator to provision a Elasticsearch cluster with one client-, one data- and one master-node.

Please refer to the upstream documentation regarding available configuration options.

To get started, put the example below into a file called elasticsearch.yaml and add it to your application's manifests. Then apply the manifests including the elasticsearch.yaml as usual.

apiVersion: enterprises.upmc.com/v1
kind: ElasticsearchCluster
metadata:
name: example-es-cluster
spec:
kibana: {}
cerebro: {}
client-node-replicas: 1
master-node-replicas: 1
data-node-replicas: 1
network-host: 0.0.0.0
zones: []
data-volume-size: 10Gi
java-options: "-Xms512m -Xmx512m"
snapshot:
scheduler-enabled: false
bucket-name: elasticsnapshots99
cron-schedule: "@every 2m"
storage:
type: standard
storage-class-version: volume.alpha.kubernetes.io/storage-class
volume-reclaim-policy: Delete