南瓜慢说知识库

# K8ssandra入门-详细记录在Linux上部署K8ssandra到Kubernetes

# 1 什么是K8ssandra

Cassandra是一款非常优秀的开源的分布式NoSQL数据库，被许多优秀的大公司采用，具有高可用、弹性扩展、性能好等特点。

正应Cassandra的优势，我们经常需要在云上服务使用，则需要部署Cassandra到K8s上，这就有了K8ssandra。K8ssandra不仅帮助我们可以快速可靠地在Kubernetes上部署Cassandra，同时提供了许多组件，如监控、备份、同步、访问等。而这些都是一个Production-Ready的产品不可或缺的。

K8ssandra的组件架构图如下：

  • Cass-operator：保证整个集群正常运行；
  • Reaper：保证一致性的同步工具；
  • Medusa：数据备份工具，支持S3、GCP Cloud Storage等；
  • Stargate：对数据访问提供API；
  • Prometheus+Grafana：云原生的常用监控组件。

# 2 安装K8ssandra

# 2.1 安装Kubenetes

如何在Ubuntu上通过Minikube快速启动一个Kubernetes，在文章《服务网格Istio入门-详细记录Kubernetes安装Istio并使用 (opens new window)》已经有详细的介绍，这里不再赘述。为了更好的兼容性，我们指定了Kubernetes的版本，命令如下：

minikube start --driver=none --kubernetes-version=v1.19.13

因为要用到PVC，我们创建一个StorageClass：

$ kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml

# 2.2 安装helm3

我们需要使用Helm来部署K8ssandra，下载Helm3如下：

# 下载安装包
curl -LO https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
# 解压
tar -zxvf helm-v3.7.0-linux-amd64.tar.gz
# 移动到bin目录
mv linux-amd64/helm /usr/local/bin/helm

添加Helm的仓库：

helm repo add k8ssandra https://helm.k8ssandra.io/stable

$ helm repo list
NAME            URL                             
k8ssandra       https://helm.k8ssandra.io/stable
traefik         https://helm.traefik.io/traefik

查找一下K8ssandra相关的包：

$ helm search repo k8ssandra
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                                       
k8ssandra/k8ssandra             1.3.1                           Provisions and configures an instance of the en...
k8ssandra/k8ssandra-common      0.28.4                          Helper library containing functions used by man...
k8ssandra/k8ssandra-operator    0.31.0          1.0.0           Kubernetes operator which handles the provision...
k8ssandra/backup                0.26.0                          Creates a CassandraBackup custom resource insta...
k8ssandra/cass-operator         0.31.0          1.8.0           Kubernetes operator which handles the provision...
k8ssandra/medusa-operator       0.30.1          0.1.0           Installs and configures the Medusa Operator for...
k8ssandra/reaper-operator       0.32.1          0.1.0           Configures and installs the Reaper Operator for...
k8ssandra/restore               0.27.1                          Creates a CassandraRestore custom resource inst...

我们安装k8ssandra/k8ssandra就可以了。

# 2.3 用Helm安装K8ssandra

Helm是一个Chart+Value的管理方式，我们准备一个yaml文件（k8ssandra-values.yaml）来放一些变量：

cassandra:
  version: "4.0.0"
  cassandraLibDirVolume:
    storageClass: local-path
    size: 5Gi
  allowMultipleNodesPerWorker: true
  heap:
    size: 1G
    newGenSize: 1G
  resources:
    requests:
      cpu: 1000m
      memory: 2Gi
    limits:
      cpu: 1000m
      memory: 2Gi
  datacenters:
  - name: dc1
    size: 1
    racks:
    - name: default 
kube-prometheus-stack:
  grafana:
    adminUser: admin
    adminPassword: admin123 
stargate:
  enabled: true
  replicas: 1
  heapMB: 256
  cpuReqMillicores: 200
  cpuLimMillicores: 1000

安装K8ssandra：

$ helm install -f k8ssandra-values.yaml k8ssandra k8ssandra/k8ssandra
NAME: k8ssandra
LAST DEPLOYED: Thu Sep 30 21:20:49 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1

做一些必要的检查如下：

$ helm list
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
k8ssandra       default         1               2021-09-30 21:20:49.409672869 +0800 CST deployed        k8ssandra-1.3.1            

$ kubectl get cassandradatacenters
NAME   AGE
dc1    4m34s

$ kubectl describe CassandraDataCenter dc1 | grep "Cassandra Operator Progress:"
  Cassandra Operator Progress:  Ready

检查一下Pod和Service：

获取K8ssandra超级用户的用户名和密码

$ kubectl get secret k8ssandra-superuser -o jsonpath="{.data.username}" | base64 --decode ; echo
k8ssandra-superuser

$ kubectl get secret k8ssandra-superuser -o jsonpath="{.data.password}" | base64 --decode ; echo
TNE5xOk45C1aQsj29qxw

# 2.4 增加节点

我们为了高可用和容量，创建更多的Cassandra Node，直接修改k8ssandra-values.yaml如下：

cassandra:
  version: "4.0.0"
  cassandraLibDirVolume:
    storageClass: local-path
    size: 5Gi
  allowMultipleNodesPerWorker: true
  heap:
    size: 1G
    newGenSize: 1G
  resources:
    requests:
      cpu: 1000m
      memory: 2Gi
    limits:
      cpu: 1000m
      memory: 2Gi
  datacenters:
  - name: dc1
    size: 3
    racks:
    - name: racks1
    - name: racks2
    - name: racks3
kube-prometheus-stack:
  grafana:
    adminUser: admin
    adminPassword: admin123 
stargate:
  enabled: true
  replicas: 1
  heapMB: 256
  cpuReqMillicores: 200
  cpuLimMillicores: 1000

修改完之后，升级配置：

$ helm upgrade -f k8ssandra-values.yaml k8ssandra k8ssandra/k8ssandra
Release "k8ssandra" has been upgraded. Happy Helming!
NAME: k8ssandra
LAST DEPLOYED: Fri Oct  1 00:40:08 2021
NAMESPACE: default
STATUS: deployed
REVISION: 2

查看Kubernetes的相关资源：

# 3 查看监控

我们对外暴露Grafana服务来看看：

kubectl expose deployment k8ssandra-grafana --type=NodePort --name=grafana-out

找到对应的NodePort的端口30348，访问：http://外网IP:30348

账号：admin/admin123

界面如下，提供良好的监控界面：

# 4 总结

K8ssandra真是一个不错的开源项目，后续再介绍如何在开发中通过K8ssandra使用Cassandra吧。

代码请查看：https://github.com/LarryDpk/pkslow-samples

Reference:

Apache Cassandra Operations in Kubernetes (opens new window)

K8ssandra Get Started (opens new window)

Install K8ssandra on GKE (opens new window)

K8ssandra Helm Chart (opens new window)

用Helm部署Kubernetes应用，支持多环境部署与版本回滚 (opens new window)

附：

使用到的镜像：

root@k8ssandra:~# docker images | sort | grep -v REPOSITORY | grep -v k8s.gcr.io
alpine                                                   3.12.2            b14afc6dfb98   9 months ago    5.57MB
busybox                                                  1.33.1            16ea53ea7c65   2 weeks ago     1.24MB
busybox                                                  latest            16ea53ea7c65   2 weeks ago     1.24MB
datastax/cass-config-builder                             1.0.4             907e52ff2f78   4 months ago    354MB
gcr.io/k8s-minikube/storage-provisioner                  v5                6e38f40d628d   6 months ago    31.5MB
grafana/grafana                                          7.3.5             71716d95fc52   9 months ago    187MB
jettech/kube-webhook-certgen                             v1.5.0            344297e197b6   11 months ago   44.7MB
k8ssandra/cass-management-api                            3.11.10-v0.1.27   e88cc3a8ce75   2 months ago    877MB
k8ssandra/cass-management-api                            4.0.0-v0.1.28     59b9869dd160   7 weeks ago     726MB
k8ssandra/cass-operator                                  v1.7.1            dd57363b0794   4 months ago    49.4MB
k8ssandra/k8ssandra-tools                                latest            43e4bd91cf9b   42 hours ago    49MB
k8ssandra/reaper-operator                                v0.3.3            b4cb9084a5e5   2 months ago    45.9MB
k8ssandra/system-logger                                  9c4c3692          4dda0db106cb   4 months ago    5.66MB
kiwigrid/k8s-sidecar                                     1.1.0             f4bb3bf1ddc7   11 months ago   82MB
quay.io/prometheus-operator/prometheus-config-reloader   v0.44.0           73bc771f8028   10 months ago   13.4MB
quay.io/prometheus-operator/prometheus-operator          v0.44.0           f5ab8d7a1a64   10 months ago   42.6MB
quay.io/prometheus/prometheus                            v2.22.1           7cc97b58fb0e   10 months ago   168MB
rancher/local-path-provisioner                           v0.0.20           933989e1174c   2 months ago    35MB
stargateio/stargate-3_11                                 v1.0.29           c276fffd9964   3 months ago    537MB
stargateio/stargate-4_0                                  v1.0.29           be4bae3748e1   3 months ago    546MB
thelastpickle/cassandra-reaper                           2.3.1             8ad2c2784ead   2 months ago    418MB
上次更新: 2023/8/18 23:39:36

← 使用Stargate访问K8ssandra，Springboot整合Cassandra →