Deploying ScyllaDB Operator with GitOps
There are many approaches to deploying applications on Kubernetes clusters. As there isn’t a clear winner, scylla-operator tries to support a wide range of deployment methods. With Scylla Operator 1.1 we brought support for helm charts (You can read more about how to use and customize them here) and since Scylla Operator 1.2 we are also publishing the manifests for deploying either manually or with GitOps automation.
What is GitOps?
GitOps is a set of practices to declaratively manage infrastructure and application configurations using Git as the single source of truth. (You can read more about GitOps in this Red Hat article.) A Git repository contains the entire state of the system making any changes visible and auditable.
In our case, the Kubernetes manifests represent the state of the system and are kept in a Git repository. Admins either apply the manifests with kubectl or use tooling that automatically reconciles the manifests from Git, like ArgoCD.
Deploying Scylla Operator from Kubernetes Manifests
In a GitOps flow, you’d copy over the manifests into an appropriate location in your own Git repository, but for the purposes of this demonstration, we’ll assume you have checked out the https://github.com/scylladb/scylla-operator repository at v1.4.0 tag and you are located at its root.
Prerequisites
Local Storage
ScyllaClusters use Kubernetes local persistent volumes exposing local disk as PersistentVolumes (PVs). There are many ways to set it up. All the operator cares about is a storage class and a matching PV being available. A common tool for mapping local disks into PVs is the Local Persistence Volume Static Provisioner. For more information have a look at our documentation. For testing purposes, you can use minikube that has an embedded dynamic storage provisioner.
(We are currently working on a managed setup by the operator, handling all this hassle for you.)
Cert-manager
Currently, the internal webhooks require the cert-manager to be installed. You can deploy it from the official manifests or use the snapshot in our repository:
$ kubectl apply -f ./examples/common/cert-manager.yaml
# Wait for the cert-manager to be ready.
$ kubectl wait --for condition=established crd/certificates.cert-manager.io crd/issuers.cert-manager.io
$ kubectl -n cert-manager rollout status deployment.apps/cert-manager-webhookDeploying the ScyllaOperator
The deployment manifests are located in ./deploy/operator and ./deploy/manager folders. For the first deployment, you’ll need to deploy the manifests in sequence as they have interdependencies, like scylla-manager needing a ScyllaCluster or establishing CRDs (propagating to all apiservers). The following instructions will get you up and running:
$ kubectl apply -f ./deploy/operator
# Wait for the operator deployment to be ready.
$ kubectl wait --for condition=established crd/scyllaclusters.scylla.scylladb.com
$ kubectl -n scylla-operator rollout status deployment.apps/scylla-operator
$ kubectl apply -f ./deploy/manager/prod
# Wait for the manager deployment to be ready.
$ kubectl -n scylla-manager rollout status deployment.apps/scylla-manager
$ kubectl -n scylla-manager rollout status deployment.apps/scylla-manager-controller
$ kubectl -n scylla-manager rollout status statefulset.apps/scylla-manager-cluster-manager-dc-manager-rackCustomization
Customization is the beauty of using Git; you can change essentially anything in a manifest and keep your changes in a patch commit. Well, changing everything is probably not the best idea to keep having a supported deployment, but changing things like the loglevel or adjusting ScyllaCluster resources for the scylla-manager makes sense.
Summary
We are constantly trying to make deploying Scylla Operator easier and manifests allow you to do that without any extra tooling with just kubectl and git, or hooking it into your GitOps automation.
For those using Operator Lifecycle Manager (OLM), we are also planning to ship an OLM bundle and publish it on the operatorhub.io so stay tuned.