Kubernetes 实战教学，手把手教您用 Helm 在 K8s 平台上部署 Prometheus

出品丨Docker公司（ID：docker-cn）
编译丨小东
每周一、三、五，与您不见不散！

---

随着单片系统变得越来越庞大以至于无法处理，许多企业开始将其分解为微服务体系结构。无论何时我们从单片架构转向微服务架构，应用程序都包含多个组件，这些组件之间的服务需要进行相互通信。每个组件都有自己的资源，可以单独扩展。如果我们谈论 Kubernetes，它会变得非常复杂，除了需要管理版本数量之外，还需要处理所有对象，例如 ConfigMaps、services、pods 以及 Persistent Volumes。 可能会出现以下挑战：

• 管理、编辑和更新多个 k8s 配置；
• 将多个 K8s 配置部署为单个应用程序；
• 共享和重用 K8s 配置和应用程序；
• 参数化并支持多个环境；
• 管理应用程序版本：rollout、rollback、diff、history；
• 定义部署生命周期（将在不同阶段运行的控制操作）；
• 部署后验证发布状态；

这些都可以通过 Kubernetes Helm 来管理，它提供了一种简单的方法来将所有东西打包到一个简单的应用程序中，并突出显示您可以配置的内容。

---

Helm 是 Kubernetes 的部署管理（不仅仅是包管理器）。它对可重复部署、依赖项管理(重用和共享)、多配置管理、更新、回滚和测试应用程序部署（版本）等方面做了大量工作。

今天，我们将在 Play with Kubernetes 平台上尝试使用 Helm。

首先，打开 https://labs.play-with-k8s.com/ 访问 Kubernetes Playground。

点击“Login”按钮以使用 Docker Hub 或 GitHub ID 进行身份验证。

一旦开始会话，您将拥有一个自己的实验室环境。

---

添加第一个 Kubernetes 节点

单击左侧的“Add New Instance” 来建您的第一个 Kubernetes 集群节点。它会自动将其命名为“node1”。每个实例都预装了Docker Community Edition（CE）和Kubeadm。该节点将被视为我们群集的主节点。

---

引导主节点

您可以通过使用以下脚本初始化主节点（node1）来引导 Kubernetes 集群。将此脚本内容复制到 bootstrap.sh 文件中，并使用“chmod + x bootstrap.sh”命令使其可执行。

当您执行此脚本时，作为初始化的一部分，kubeadm 将会编写所需的几个配置文件，设置 RBAC 并部署Kubernetes控制平面组件（如 kube-apiserver、kube-dns、kube-proxy 和 etcd等等）。控制平面组件以 Docker 容器的形式进行部署。

复制上面的 kubeadm 连接令牌命令，并将其保存以供下一步使用。此命令将用于将其他节点连接到集群。

---

添加工作节点

点击“Add New Node”来添加一个新的工作节点。

---

检查集群状态

[node1 ~]$ kubectl get nodes NAME STATUS ROLES AGE VERSION node1 Ready master 18m v1.11.3 node2 Ready 4m v1.11.3 node3 Ready 39s v1.11.3 node4 NotReady 22s v1.11.3 node5 NotReady 4s v1.11.3 [node1 ~]$ [node1 ]$ kubectl get po No resources found. [node1 ]$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 443/TCP 1h [node1]$

---

验证正在运行的 Pod

[node1 ~]$ kubectl get nodes -o json | > jq ".items[] | {name:.metadata.name} + .status.capacity" { "name": "node1", "cpu": "8", "ephemeral-storage": "10Gi", "hugepages-1Gi": "0", "hugepages-2Mi": "0", "memory": "32929612Ki", "pods": "110" } { "name": "node2", "cpu": "8", "ephemeral-storage": "10Gi", "hugepages-1Gi": "0", "hugepages-2Mi": "0", "memory": "32929612Ki", "pods": "110" } { "name": "node3", "cpu": "8", "ephemeral-storage": "10Gi", "hugepages-1Gi": "0", "hugepages-2Mi": "0", "memory": "32929612Ki", "pods": "110" } { "name": "node4", "cpu": "8", "ephemeral-storage": "10Gi", "hugepages-1Gi": "0", "hugepages-2Mi": "0", "memory": "32929612Ki", "pods": "110" } { "name": "node5", "cpu": "8", "ephemeral-storage": "10Gi", "hugepages-1Gi": "0", "hugepages-2Mi": "0", "memory": "32929612Ki", "pods": "110" } 

---

安装 OpenSSL

[node1 ~]$ yum install -y openssl 

---

安装 Helm

$ url https://raw.githubusercontent.com/helm/helm/master/scripts/get > get_helm.sh $ chmod 700 get_helm.sh $ ./get_helm.sh [node1 ~]$ sh get_helm.sh Downloading https://kubernetes-helm.storage.googleapis.com/helm-v2.11.0-linux-amd64.tar.gz Preparing to install helm and tiller into /usr/local/bin helm installed into /usr/local/bin/helm tiller installed into /usr/local/bin/tiller get_helm.sh: line 177: which: command not found Run 'helm init' to configure helm. [node1 ~]$ helm init Creating /root/.helm Creating /root/.helm/repository Creating /root/.helm/repository/cache Creating /root/.helm/repository/local Creating /root/.helm/plugins Creating /root/.helm/starters Creating /root/.helm/cache/archive Creating /root/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com Adding local repo with URL: http://127.0.0.1:8879/charts $HELM_HOME has been configured at /root/.helm. Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster. Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy. To prevent this, run `helm init` with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation Happy Helming 

---

安装 Prometheus

让我们尝试使用 Helm 在拥有 5 个节点 K8s 群集上安装 Prometheus Stack。

首先，可以使用“helm search ”选项搜索应用程序堆栈。

[node1 ~]$ helm search prometheus NAME CHART VERSION APP VERSION DESCRIPTION stable/prometheus 7.3.4 2.4.3 Prometheus is a monitoring system and time series database. stable/prometheus-adapter v0.2.0 v0.2.1 A Helm chart for k8s prometheus adapter stable/prometheus-blackbox-exporter 0.1.3 0.12.0 Prometheus Blackbox Exporter stable/prometheus-cloudwatch-exporter 0.2.1 0.5.0 A Helm chart for prometheus cloudwatch-exporter stable/prometheus-couchdb-exporter 0.1.0 1.0 A Helm chart to export the metrics from couchdb in Promet... stable/prometheus-mysql-exporter 0.2.1 v0.11.0 A Helm chart for prometheus mysql exporter with cloudsqlp... stable/prometheus-node-exporter 0.5.0 0.16.0 A Helm chart for prometheus node-exporter stable/prometheus-operator 0.1.7 0.24.0 Provides easy monitoring definitions for Kubernetes servi... stable/prometheus-postgres-exporter 0.5.0 0.4.6 A Helm chart for prometheus postgres-exporter stable/prometheus-pushgateway 0.1.3 0.6.0 A Helm chart for prometheus pushgateway stable/prometheus-rabbitmq-exporter 0.1.4 v0.28.0 Rabbitmq metrics exporter for prometheus stable/prometheus-redis-exporter 0.3.2 0.21.1 Prometheus exporter for Redis metrics stable/prometheus-to-sd 0.1.1 0.2.2 Scrape metrics stored in prometheus format and push them ... stable/elasticsearch-exporter 0.4.0 1.0.2 Elasticsearch stats exporter for Prometheus stable/karma 1.1.2 v0.14 A Helm chart for Karma - an UI for Prometheus Alertmanager stable/stackdriver-exporter 0.0.4 0.5.1 Stackdriver exporter for Prometheus stable/weave-cloud 0.3.0 1.1.0 Weave Cloud is a add-on to Kubernetes which provides Cont... stable/kube-state-metrics 0.9.0 1.4.0 Install kube-state-metrics to generate and expose cluster... stable/mariadb 5.2.2 10.1.36 Fast, reliable, scalable, and easy to use open-source rel... [node1 ~]$ 

---

更新镜像仓库

[node1 ~]$ helm repo update Hang tight while we grab the latest from your chart repositories... ...Skip local chart repository ...Successfully got an update from the "stable" chart repository Update Complete. ⎈ Happy Helming!⎈

---

安装 Prometheus

$helm install stable/prometheus 

出现错误提示：“default” is forbidden: User “system:serviceaccount:kube-system:default” cannot get namespaces in the namespace “default”

---

如何修复？

要解决该问题，您需要按照以下步骤操作：

kubectl --namespace kube-system create serviceaccount tiller kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller helm init --service-account tiller --upgrade 

---

Helm 清单

[node1 ~]$ helm list NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE excited-elk 1 Sun Oct 28 10:00:02 2018 DEPLOYED prometheus-7.3.4 2.4.3 default [node1 ~]$ helm repo update Hang tight while we grab the latest from your chart repositories... ...Skip local chart repository ...Successfully got an update from the "stable" chart repository Update Complete. ⎈ Happy Helming!⎈ [node1 ~]$ helm install stable/prometheus NAME: excited-elk LAST DEPLOYED: Sun Oct 28 10:00:02 2018 NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1beta1/DaemonSet NAME AGE excited-elk-prometheus-node-exporter 1s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE excited-elk-prometheus-node-exporter-7bjqc 0/1 ContainerCreating 0 1s excited-elk-prometheus-node-exporter-gbcd7 0/1 ContainerCreating 0 1s excited-elk-prometheus-node-exporter-tk56q 0/1 ContainerCreating 0 1s excited-elk-prometheus-node-exporter-tkk9b 0/1 ContainerCreating 0 1s excited-elk-prometheus-alertmanager-68f4f57c97-wrfjz 0/2 Pending 0 1s excited-elk-prometheus-kube-state-metrics-858d44dfdc-vt4wj 0/1 ContainerCreating 0 1s excited-elk-prometheus-pushgateway-58bfd54d6d-m4n69 0/1 ContainerCreating 0 1s excited-elk-prometheus-server-5958586794-b97xn 0/2 Pending 0 1s ==> v1/ConfigMap NAME AGE excited-elk-prometheus-alertmanager 1s excited-elk-prometheus-server 1s ==> v1/ServiceAccount excited-elk-prometheus-alertmanager 1s excited-elk-prometheus-kube-state-metrics 1s excited-elk-prometheus-node-exporter 1s excited-elk-prometheus-pushgateway 1s excited-elk-prometheus-server 1s ==> v1beta1/ClusterRole excited-elk-prometheus-kube-state-metrics 1s excited-elk-prometheus-server 1s ==> v1beta1/Deployment excited-elk-prometheus-alertmanager 1s excited-elk-prometheus-kube-state-metrics 1s excited-elk-prometheus-pushgateway 1s excited-elk-prometheus-server 1s ==> v1/PersistentVolumeClaim excited-elk-prometheus-alertmanager 1s excited-elk-prometheus-server 1s ==> v1beta1/ClusterRoleBinding excited-elk-prometheus-kube-state-metrics 1s excited-elk-prometheus-server 1s ==> v1/Service excited-elk-prometheus-alertmanager 1s excited-elk-prometheus-kube-state-metrics 1s excited-elk-prometheus-node-exporter 1s excited-elk-prometheus-pushgateway 1s excited-elk-prometheus-server 1s NOTES: The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster: excited-elk-prometheus-server.default.svc.cluster.local Get the Prometheus server URL by running these commands in the same shell: export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}") kubectl --namespace default port-forward $POD_NAME 9090 The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster: excited-elk-prometheus-alertmanager.default.svc.cluster.local Get the Alertmanager URL by running these commands in the same shell: export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=alertmanager" -o jsonpath="{.items[0].metadata.name}") kubectl --namespace default port-forward $POD_NAME 9093 The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster: excited-elk-prometheus-pushgateway.default.svc.cluster.local Get the PushGateway URL by running these commands in the same shell: export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}") kubectl --namespace default port-forward $POD_NAME 9091 For more information on running Prometheus, visit: https://prometheus.io/ [node1 ~]$ kubectl get all NAME READY STATUS RESTARTS AGE pod/excited-elk-prometheus-alertmanager-68f4f57c97-wrfjz 0/2 Pending 0 3m pod/excited-elk-prometheus-kube-state-metrics-858d44dfdc-vt4wj 1/1 Running 0 3m pod/excited-elk-prometheus-node-exporter-7bjqc 1/1 Running 0 3m pod/excited-elk-prometheus-node-exporter-gbcd7 1/1 Running 0 3m pod/excited-elk-prometheus-node-exporter-tk56q 1/1 Running 0 3m pod/excited-elk-prometheus-node-exporter-tkk9b 1/1 Running 0 3m pod/excited-elk-prometheus-pushgateway-58bfd54d6d-m4n69 1/1 Running 0 3m pod/excited-elk-prometheus-server-5958586794-b97xn 0/2 Pending 0 3m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/excited-elk-prometheus-alertmanager ClusterIP 10.106.159.46 80/TCP 3m service/excited-elk-prometheus-kube-state-metrics ClusterIP None 80/TCP 3m service/excited-elk-prometheus-node-exporter ClusterIP None 9100/TCP 3m service/excited-elk-prometheus-pushgateway ClusterIP 10.106.88.15 9091/TCP 3m service/excited-elk-prometheus-server ClusterIP 10.107.15.64 80/TCP 3m service/kubernetes ClusterIP 10.96.0.1 443/TCP 37m NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/excited-elk-prometheus-node-exporter 4 4 4 4 4 3m NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/excited-elk-prometheus-alertmanager 1 1 1 0 3m deployment.apps/excited-elk-prometheus-kube-state-metrics 1 1 1 1 3m deployment.apps/excited-elk-prometheus-pushgateway 1 1 1 1 3m deployment.apps/excited-elk-prometheus-server 1 1 1 0 3m NAME DESIRED CURRENT READY AGE replicaset.apps/excited-elk-prometheus-alertmanager-68f4f57c97 1 1 0 3m replicaset.apps/excited-elk-prometheus-kube-state-metrics-858d44dfdc 1 1 1 3m replicaset.apps/excited-elk-prometheus-pushgateway-58bfd54d6d 1 1 1 3m replicaset.apps/excited-elk-prometheus-server-5958586794 1 1 0 3m [node1 ~]$ 

等待几分钟，您就可以使用 https://:9090 访问 Prometheus UI了。

---

在下一篇 Kubernetes 实战教学系列文章中，我将为您带来更多关于 Helm on PWD Playground 的有趣内容。