This project is forked and adapted from Nuance-Mobility/zookeeper-operator
The ZooKeeper Operator manages zookeeper clusters deployed to Kubernetes and automates tasks related to operating a ZooKeeper cluster.
- Kubernetes 1.15+
- ZooKeeper 3.5.0+
Refer to BUILDING to build zookeeper-operator images.
Create a deployment for ZooKeeper Operator:
$ kubectl create -f example/deployment.yamlZooKeeper Operator will automatically create a Kubernetes Custom Resource Definition (CRD):
$ kubectl get customresourcedefinitions
NAME AGE
zookeeperclusters.zookeeper.database.apache.com 1mNote that the ZooKeeper clusters managed by ZooKeeper Operator will NOT be deleted even if the Operator is uninstalled.
This is an intentional design to prevent accidental Operator failure from killing all the ZooKeeper clusters.
To delete all clusters, delete all cluster CR objects before uninstalling the Operator.
Clean up ZooKeeper Operator:
-
Delete the Operator.
$ kubectl delete -f example/deployment.yaml
-
Clean related resources.
$ kubectl api-resources --verbs=get -o name | grep -v componentstatus | xargs -n 1 kubectl get --show-kind --ignore-not-found -l app=zookeeper-operator -o name | xargs kubectl delete
Refer to BUILDING to build zookeeper-instance images.
Create a ZooKeeper cluster:
$ kubectl create -f example/example-zookeeper-cluster.yamlA 3 member ZooKeeper cluster will be created.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-zookeeper-cluster-1 1/1 Running 0 1m
example-zookeeper-cluster-2 1/1 Running 0 1m
example-zookeeper-cluster-3 1/1 Running 0 1mDestroy ZooKeeper cluster:
$ kubectl delete -f example/example-zookeeper-cluster.yamlCreate a ZooKeeper cluster:
$ kubectl apply -f example/example-zookeeper-cluster.yamlIn example/example-zookeeper-cluster.yaml the initial cluster size is 3.
Modify the file and change size from 3 to 5.
$ cat example/example-zookeeper-cluster.yaml
apiVersion: "zookeeper.database.apache.com/v1alpha1"
kind: "ZooKeeperCluster"
metadata:
name: "example-zookeeper-cluster"
spec:
size: 5
version: "3.5.7"Apply the size change to the cluster CR:
$ kubectl apply -f example/example-zookeeper-cluster.yamlThe ZooKeeper cluster will scale to 5 members (5 pods):
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-zookeeper-cluster-1 1/1 Running 0 1m
example-zookeeper-cluster-2 1/1 Running 0 1m
example-zookeeper-cluster-3 1/1 Running 0 1m
example-zookeeper-cluster-4 1/1 Running 0 1m
example-zookeeper-cluster-5 1/1 Running 0 1mSimilarly we can decrease the size of cluster from 5 back to 3 by changing the size field again and reapplying the change.
$ cat example/example-zookeeper-cluster.yaml
apiVersion: "zookeeper.database.apache.com/v1alpha1"
kind: "ZooKeeperCluster"
metadata:
name: "example-zookeeper-cluster"
spec:
size: 3
version: "3.5.7"$ kubectl apply -f example/example-zookeeper-cluster.yamlWe should see that ZooKeeper cluster will eventually reduce to 3 pods:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-zookeeper-cluster-2 1/1 Running 0 1m
example-zookeeper-cluster-3 1/1 Running 0 1m
example-zookeeper-cluster-5 1/1 Running 0 1mIf the minority of ZooKeeper members crash, the ZooKeeper Operator will automatically recover the failure. Let's walk through in the following steps.
Create a ZooKeeper cluster:
$ kubectl create -f example/example-zookeeper-cluster.yamlWait until all three members are up. Simulate a member failure by deleting a pod:
$ kubectl delete pod example-zookeeper-cluster-1 --nowThe ZooKeeper Operator will recover the failure by creating a new pod example-zookeeper-cluster-4:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-zookeeper-cluster-2 1/1 Running 0 1m
example-zookeeper-cluster-3 1/1 Running 0 1m
example-zookeeper-cluster-4 1/1 Running 0 1mDestroy ZooKeeper cluster:
$ kubectl delete -f example/example-zookeeper-cluster.yamlIf the ZooKeeper Operator restarts, it can recover its previous state. Let's walk through in the following steps.
$ kubectl create -f example/example-zookeeper-cluster.yamlWait until all three members are up. Then
$ kubectl delete -f example/deployment.yaml
deployment "zookeeper-operator" deleted
$ kubectl delete pod example-zookeeper-cluster-1 --now
pod "example-zookeeper-cluster-1" deletedThen restart the ZooKeeper Operator. It should recover itself and the ZooKeeper clusters it manages.
$ kubectl create -f example/deployment.yaml
deployment "zookeeper-operator" created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-zookeeper-cluster-2 1/1 Running 0 1m
example-zookeeper-cluster-3 1/1 Running 0 1m
example-zookeeper-cluster-4 1/1 Running 0 1mRefer to BUILDING to build zookeeper-instance images.
Prerequisition: we need to have an old and a new zookeeper-instance image.
$ docker images| grep zookeeper-instance
zookeeper-instance 3.5.8 09d22b7c9006 About a minute ago 245MB
zookeeper-instance 3.5.7 89fb69febb56 7 weeks ago 245MBCreate a ZooKeeper cluster:
$ kubectl apply -f example/example-zookeeper-cluster.yaml$ kubectl describe pod/example-zookeeper-cluster-0| grep Image| grep zookeeper-instance
Image: zookeeper-instance:3.5.7In example/example-zookeeper-cluster.yaml the initial zookeeper version is "3.5.7".
Modify the file and change version from "3.5.7" to "3.5.8".
$ cat example/example-zookeeper-cluster.yaml
apiVersion: "zookeeper.database.apache.com/v1alpha1"
kind: "ZooKeeperCluster"
metadata:
name: "example-zookeeper-cluster"
spec:
size: 3
version: "3.5.8"Apply the change to the cluster CR:
$ kubectl apply -f example/example-zookeeper-cluster.yamlThe ZooKeeper cluster will be upgraded to version "3.5.8":
$ kubectl describe pod/example-zookeeper-cluster-0| grep Image| grep zookeeper-instance
Image: zookeeper-instance:3.5.8Refer to future-plans