# kubeadmin 安装k8s1.20集群 标签（空格分隔）： kubernetes架构系列 --- [toc] ---- ## 一：k8s1.20.x 的重要更新 ``` 1、Kubectl debug 设置一个临时容器 2、Sidecar 3、Volume：更改目录权限，fsGroup 4、ConfigMap和Secret K8S官网：https://kubernetes.io/docs/setup/ 最新版高可用安装：https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/ ``` ------ ## 二：k8s1.20.x 的安装 ### 2.1 ：高可用Kubernetes集群规划 ![image_1f0sgr8qq1d2inqr35ml3g1qsu9.png-43.5kB][1] ``` k8s 的高可用的架构图 ``` ![image_1f1m9lqt31do01d3t1rgejtbfnn9.png-131.5kB][2] ``` 所有节点配置hosts，修改/etc/hosts如下： cat /etc/hosts ---- 192.168.100.11 node01.flyfish.cn 192.168.100.12 node02.flyfish.cn 192.168.100.13 node03.flyfish.cn 192.168.100.14 node04.flyfish.cn 192.168.100.15 node05.flyfish.cn 192.168.100.16 node06.flyfish.cn 192.168.100.17 node07.flyfish.cn 192.168.100.18 node08.flyfish.cn ---- ``` ----- ### 2.2 yum 的更新配置 （所有节点全部安装） ``` curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo ``` ---- ![image_1f0shlio9u63h1ct111vne154m.png-246.8kB][3] --- ``` 必备工具安装: yum install wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 git -y ``` ![image_1f0shp7p44mc1vit1cf1grd1i2s13.png-120.7kB][4] ---- ``` 所有节点关闭防火墙、selinux、dnsmasq、swap。服务器配置如下： systemctl disable --now firewalld systemctl disable --now dnsmasq systemctl disable --now NetworkManager setenforce 0 sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/sysconfig/selinux sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config ``` ---- ``` 关闭swap分区 (全部节点) swapoff -a && sysctl -w vm.swappiness=0 sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab ``` ![image_1f0sht5fipl21jlj9ck1ljlt8a1g.png-58.1kB][5] ---- ``` 安装ntpdate rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm yum install ntpdate -y 所有节点同步时间。时间同步配置如下： ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime echo 'Asia/Shanghai' >/etc/timezone ntpdate time2.aliyun.com 加入到crontab */5 * * * * ntpdate time2.aliyun.com 所有节点配置limit： ulimit -SHn 65535 vim /etc/security/limits.conf # 末尾添加如下内容 * soft nofile 655360 * hard nofile 131072 * soft nproc 655350 * hard nproc 655350 * soft memlock unlimited * hard memlock unlimited ``` ---- ``` 安装ntpdate rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm yum install ntpdate -y 所有节点同步时间。时间同步配置如下： ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime echo 'Asia/Shanghai' >/etc/timezone ntpdate time2.aliyun.com 加入到crontab */5 * * * * ntpdate time2.aliyun.com 所有节点配置limit： ulimit -SHn 65535 vim /etc/security/limits.conf # 末尾添加如下内容 * soft nofile 655360 * hard nofile 131072 * soft nproc 655350 * hard nproc 655350 * soft memlock unlimited * hard memlock unlimited Master01节点免密钥登录其他节点： ssh-keygen -t rsa for i in k8s-master01.flyfish.cn k8s-master02.flyfish.cn k8s-master03.flyfish.cn k8s-node01.flyfish.cn k8s-node02.flyfish.cn;do ssh-copy-id -i .ssh/id_rsa.pub $i;done 所有节点升级系统并重启： yum update -y && reboot ``` ----- ``` 下载安装源码文件： cd /root/ ; git clone https://github.com/dotbalo/k8s-ha-install.git ``` ---- ``` CentOS 7安装yum源如下： curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo ``` ---- ``` CentOS 8 安装源如下： curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo ``` ---- ``` 所有节点升级系统并重启，此处升级没有升级内核，下节会单独升级内核： yum install wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 -y yum update -y --exclude=kernel* && reboot #CentOS7需要升级，8不需要 ``` ----- ``` 1.1.2 内核配置 CentOS7 需要升级内核至4.18+ https://www.kernel.org/ 和 https://elrepo.org/linux/kernel/el7/x86_64/ CentOS 7 dnf可能无法安装内核 dnf --disablerepo=\* --enablerepo=elrepo -y install kernel-ml kernel-ml-devel grubby --default-kernel 使用如下方式安装最新版内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm 查看最新版内核yum --disablerepo="*" --enablerepo="elrepo-kernel" list available [root@k8s-node01 ~]# yum --disablerepo="*" --enablerepo="elrepo-kernel" list available Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * elrepo-kernel: mirrors.neusoft.edu.cn elrepo-kernel | 2.9 kB 00:00:00 elrepo-kernel/primary_db | 1.9 MB 00:00:00 Available Packages elrepo-release.noarch 7.0-5.el7.elrepo elrepo-kernel kernel-lt.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-devel.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-doc.noarch 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-headers.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-tools.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-tools-libs.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-lt-tools-libs-devel.x86_64 4.4.229-1.el7.elrepo elrepo-kernel kernel-ml.x86_64 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-devel.x86_64 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-doc.noarch 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-headers.x86_64 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-tools.x86_64 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-tools-libs.x86_64 5.7.7-1.el7.elrepo elrepo-kernel kernel-ml-tools-libs-devel.x86_64 5.7.7-1.el7.elrepo elrepo-kernel perf.x86_64 5.7.7-1.el7.elrepo elrepo-kernel python-perf.x86_64 5.7.7-1.el7.elrepo elrepo-kernel 安装最新版： yum --enablerepo=elrepo-kernel install kernel-ml kernel-ml-devel –y 安装完成后reboot 更改内核顺序： grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg && grubby --args="user_namespace.enable=1" --update-kernel="$(grubby --default-kernel)" && reboot 开机后查看内核 [appadmin@k8s-node01 ~]$ uname -a Linux k8s-node01 5.7.7-1.el7.elrepo.x86_64 #1 SMP Wed Jul 1 11:53:16 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux CentOS 8按需升级: 可以采用dnf升级，也可使用上述同样步骤升级（使用上述步骤注意elrepo-release-8.1版本） rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org dnf install https://www.elrepo.org/elrepo-release-8.1-1.el8.elrepo.noarch.rpm dnf --disablerepo=\* --enablerepo=elrepo -y install kernel-ml kernel-ml-devel grubby --default-kernel && reboot ``` ---- ``` 安装依赖包： 本所有节点安装ipvsadm： yum install ipvsadm ipset sysstat conntrack libseccomp -y 所有节点配置ipvs模块，在内核4.19+版本nf_conntrack_ipv4已经改为nf_conntrack，本例安装的内核为4.18，使用nf_conntrack_ipv4即可： modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 cat /etc/modules-load.d/ipvs.conf ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack_ipv4 ip_tables ip_set xt_set ipt_set ipt_rpfilter ipt_REJECT ipip 然后执行systemctl enable --now systemd-modules-load.service即可 ``` ----- ``` 开启一些k8s集群中必须的内核参数，所有节点配置k8s内核： cat < /etc/sysctl.d/k8s.conf net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-iptables = 1 fs.may_detach_mounts = 1 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 fs.file-max=52706963 fs.nr_open=52706963 net.netfilter.nf_conntrack_max=2310720 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_keepalive_intvl =15 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_max_orphans = 327680 net.ipv4.tcp_orphan_retries = 3 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.ip_conntrack_max = 65536 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.tcp_timestamps = 0 net.core.somaxconn = 16384 EOF sysctl --system ``` ----- ``` 1.1.3 基本组件安装 本节主要安装的是集群中用到的各种组件，比如Docker-ce、Kubernetes各组件等。 查看可用docker-ce版本： yum list docker-ce.x86_64 --showduplicates | sort -r [root@k8s-master01 k8s-ha-install]# wget https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.13-3.2.el7.x86_64.rpm 安装 docker-ce 19.03 版本： yum install -y docker-ce-cli-19.03.8-3.el7.x86_64 docker-ce-19.03.8-3.el7.x86_64 ``` ``` 温馨提示： 由于新版kubelet建议使用systemd，所以可以把docker的CgroupDriver改成systemd cat > /etc/docker/daemon.json < /etc/sysconfig/kubelet< > /root/.bashrc export KUBECONFIG=/etc/kubernetes/admin.conf EOF source /root/.bashrc 查看节点状态： [root@k8s-master01 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master01 NotReady master 14m v1.12.3 采用初始化安装方式，所有的系统组件均以容器的方式运行并且在kube-system命名空间内，此时可以查看Pod状态： ``` ---- ``` 其他master 加入集群： kubeadm join 192.168.100.200:16443 --token 7t2weq.bjbawausm0jaxury \ --discovery-token-ca-cert-hash sha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99 \ --control-plane --certificate-key c0b3b67c42f4fe9ae2832d86f80df35ee2e7b32f945906fabe60e4ae1f4ba18f ``` ![image_1f1ms7272ufi33bcm1bit1b7e4q.png-274.5kB][18] ``` mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` ---- ``` 加master节点报错 [ERROR DirAvailable--var-lib-etcd]: /var/lib/etcd is not empty [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher ``` ![image_1f1msjrle1p9817bd1u2etq0i1u57.png-115kB][19] ``` rm -rf /var/lib/etcd ``` ---- ``` 在初始化： kubeadm join 192.168.100.200:16443 --token 7t2weq.bjbawausm0jaxury \ --discovery-token-ca-cert-hash sha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99 \ --control-plane --certificate-key c0b3b67c42f4fe9ae2832d86f80df35ee2e7b32f945906fabe60e4ae1f4ba18f ``` ![image_1f1msni0f1vnbj3qvkeige1i6k5k.png-371.2kB][20] ``` mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` ![image_1f1msorv8stapetkm8nfm1ejp61.png-119.5kB][21] ``` kubectl get node ``` ![image_1f1msplr71chm105p5k5bkfq0i6e.png-72.6kB][22] ---- ``` node 节点加入： kubeadm join 192.168.100.200:16443 --token 7t2weq.bjbawausm0jaxury \ --discovery-token-ca-cert-hash sha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99 ``` ![image_1f1msu790akc19nqrce57k1lnr6r.png-129.4kB][23] ---- ``` Token过期后生成新的token：(集群扩容与缩容的问题) kubeadm token create --print-join-command ----- kubeadm join 192.168.100.200:16443 --token p6rvkq.0joqbi5bxnd12n20 --discovery-token-ca-cert-hash sha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99 ----- Master需要生成--certificate-key kubeadm init phase upload-certs --upload-certs ----- [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace [upload-certs] Using certificate key: c97af03609a9d3c099dd07615ce170a8f0c8db368f36e10a79576f9efcc0857e ---- 初始化其他master加入集群 kubeadm join 192.168.100.200:16443 --token p6rvkq.0joqbi5bxnd12n20 --discovery-token-ca-cert-hash sha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99 --control-plane --certificate-key c97af03609a9d3c099dd07615ce170a8f0c8db368f36e10a79576f9efcc0857e ``` ----- ![image_1f1mu5m9711nt11o4g6r1p9u1j7878.png-103.8kB][24] ``` 查看token的过期时间 kubectl get secret -n kube-system -o wide ``` ---- ``` kubectl get secret -n kube-system bootstrap-token-q8soir -o yaml ----- apiVersion: v1 data: description: UHJveHkgZm9yIG1hbmFnaW5nIFRUTCBmb3IgdGhlIGt1YmVhZG0tY2VydHMgc2VjcmV0 expiration: MjAyMS0wMy0yNlQxODoyMzowNSswODowMA== token-id: cThzb2ly token-secret: cnZidjZwdGIxajIxbWFsZw== kind: Secret metadata: creationTimestamp: "2021-03-26T08:23:05Z" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:data: .: {} f:description: {} f:expiration: {} f:token-id: {} f:token-secret: {} f:type: {} manager: kubeadm operation: Update time: "2021-03-26T08:23:05Z" name: bootstrap-token-q8soir namespace: kube-system resourceVersion: "305" uid: eba712c8-386e-4824-802e-7a2f5cd05f0c type: bootstrap.kubernetes.io/token ---- ``` ![image_1f1mug5h37741ib913ruksd170g7l.png-158.2kB][25] ------- ``` echo MjAyMS0wMy0yNlQxODoyMzowNSswODowMA== |base64 -d ``` ![image_1f1muher87hf4ebbs718ak17cp82.png-196kB][26] ---- ``` kubectl get node ``` ![image_1f1muodoa1anri996om1eslnro8f.png-73.9kB][27] ``` 修改calico-etcd.yaml的以下位置 cd /root/k8s-ha-install && git checkout manual-installation-v1.20.x && cd calico/ ----- sed -i 's#etcd_endpoints: "http:// : "#etcd_endpoints: "https://192.168.100.11:2379,https://192.168.100.12:2379,https://192.168.100.13:2379"#g' calico-etcd.yaml ETCD_CA=`cat /etc/kubernetes/pki/etcd/ca.crt | base64 | tr -d '\n'` ETCD_CERT=`cat /etc/kubernetes/pki/etcd/server.crt | base64 | tr -d '\n'` ETCD_KEY=`cat /etc/kubernetes/pki/etcd/server.key | base64 | tr -d '\n'` sed -i "s@# etcd-key: null@etcd-key: ${ETCD_KEY}@g; s@# etcd-cert: null@etcd-cert: ${ETCD_CERT}@g; s@# etcd-ca: null@etcd-ca: ${ETCD_CA}@g" calico-etcd.yaml sed -i 's#etcd_ca: ""#etcd_ca: "/calico-secrets/etcd-ca"#g; s#etcd_cert: ""#etcd_cert: "/calico-secrets/etcd-cert"#g; s#etcd_key: "" #etcd_key: "/calico-secrets/etcd-key" #g' calico-etcd.yaml POD_SUBNET=`cat /etc/kubernetes/manifests/kube-controller-manager.yaml | grep cluster-cidr= | awk -F= '{print $NF}'` sed -i 's@# - name: CALICO_IPV4POOL_CIDR@- name: CALICO_IPV4POOL_CIDR@g; s@# value: "192.168.0.0/16"@ value: '"${POD_SUBNET}"'@g' calico-etcd.yaml ----- kubectl apply -f calico-etcd.yaml kubectl get node -n kue-system kubectl get node ``` ![image_1f1n1am3q1evr1lnh1eti1l0c166a8s.png-201.3kB][28] ![image_1f1n1bivn15lkptmvda15sk1f1999.png-65.3kB][29] ``` 配置metric server cd /root/k8s-ha-install/metrics-server-0.4.x-kubeadm vim comp.ymal ---- apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server rbac.authorization.k8s.io/aggregate-to-admin: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-view: "true" name: system:aggregated-metrics-reader rules: - apiGroups: - metrics.k8s.io resources: - pods - nodes verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - nodes/stats - namespaces - configmaps verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: v1 kind: Service metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: ports: - name: https port: 443 protocol: TCP targetPort: https selector: k8s-app: metrics-server --- apiVersion: apps/v1 kind: Deployment metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: selector: matchLabels: k8s-app: metrics-server strategy: rollingUpdate: maxUnavailable: 0 template: metadata: labels: k8s-app: metrics-server spec: containers: - args: - --cert-dir=/tmp - --secure-port=4443 - --metric-resolution=30s - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname # - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt # change to front-proxy-ca.crt for kubeadm - --requestheader-username-headers=X-Remote-User - --requestheader-group-headers=X-Remote-Group - --requestheader-extra-headers-prefix=X-Remote-Extra- image: registry.cn-beijing.aliyuncs.com/dotbalo/metrics-server:v0.4.1 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /livez port: https scheme: HTTPS periodSeconds: 10 name: metrics-server ports: - containerPort: 4443 name: https protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /readyz port: https scheme: HTTPS periodSeconds: 10 securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 volumeMounts: - mountPath: /tmp name: tmp-dir - name: ca-ssl mountPath: /etc/kubernetes/pki nodeSelector: kubernetes.io/os: linux priorityClassName: system-cluster-critical serviceAccountName: metrics-server volumes: - emptyDir: {} name: tmp-dir - name: ca-ssl hostPath: path: /etc/kubernetes/pki --- apiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: labels: k8s-app: metrics-server name: v1beta1.metrics.k8s.io spec: group: metrics.k8s.io groupPriorityMinimum: 100 insecureSkipTLSVerify: true service: name: metrics-server namespace: kube-system version: v1beta1 versionPriority: 100 ---- kubectl apply -f comp.yaml ``` ![image_1f9m4197m14vmmup1tfiede183d9.png-140.2kB][30] ![image_1f9m42fcr15681hpi1cmn7st1d59m.png-253.4kB][31] --- ``` 安装dashbaord cd /root/k8s-ha-install/dashboard kubectl apply -f dashboard-user.yaml kubectl apply -f dashboard.yaml kubectl get pod -n kubernetes-dashboard ``` ![image_1f9m4bogl9g21v231tc16aev0e13.png-133.5kB][32] ``` kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard 改一下svc 的 类型： type: Cluster-IP 改为： type: NodePort ``` ![image_1f9m4p2ed167q8r51vbr1n83lbj1g.png-36.5kB][33] ``` kubectl get svc -n kubernetes-dashboard -o wide ``` ![image_1f9m4qfrlvign6k1venf2p12gj2d.png-105.9kB][34] ``` kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}') ``` ![image_1f9m4tjr7jnbd7p1rlk1lv911n5a.png-235.4kB][35] ![image_1f9m4vn2219281hupojg8or14le5n.png-271kB][36] ![image_1f9m52b078d91ei3p0cu351eqp8k.png-450kB][37] ![image_1f9m64ouv1slimm81t5k1f5u15r391.png-379.7kB][38] ``` 集群测试： kubectl get svc -n kube-system telnet 10.96.0.1 443 telnet 10.96.0.10 53 ``` ![image_1f9mf904t1grm1akf1nt51b1v1g879.png-103.6kB][39] ![image_1f9mflqut1moj1dqh9ojbko1pju26.png-180.5kB][40] ``` 部署kuborad 在node 节点上面 下载镜像： docker pull eipwork/kuboard:latest kubectl apply -f https://kuboard.cn/install-script/kuboard.yaml kubectl get pods -l k8s.kuboard.cn/name=kuboard -n kube-system kubectl get svc,pod -n kube-system -o wide |grep kuboard ``` ![image_1f9n6n32h1iko1st01r792si9fa9.png-80.8kB][41] ![image_1f9n6tps337kp864141oq21v9p13.png-93.3kB][42] ``` 获取Token # 如果您参考 www.kuboard.cn 提供的文档安装 Kuberenetes，可在第一个 Master 节点上执行此命令 echo $(kubectl -n kube-system get secret $(kubectl -n kube-system get secret | grep kuboard-user | awk '{print $1}') -o go-template='{{.data.token}}' | base64 -d) ``` ![image_1f9n72ituq841b7nfrkare1jlp1g.png-188.7kB][43] ![image_1f9n73hqf1a24qkjsf37ms12et1t.png-406.6kB][44] ![image_1f9n74aga151hkhfknnhlts3n2a.png-563.5kB][45] ---- ``` 卸载：kuborad-v2 kubectl delete -f https://kuboard.cn/install-script/kuboard.yaml ``` ----- ``` 安装kuboard-v3 在node 节点上面下载镜像： docker pull eipwork/kuboard:v3 docker pull eipwork/etcd-host:3.4.16-1 mkdir /data chmod 777 -R /data 配置镜像下载策略 wget https://addons.kuboard.cn/kuboard/kuboard-v3.yaml vim kuboard-v3.ymal --- imagePullPolicy: IfNotPresent (共有两处) --- kubectl apply -f kuboard-v3.yaml ``` ![image_1f9n87u3i12901tq0ad71ku71er52n.png-95.4kB][46] ![image_1f9n89h5e1sbs4h41pf7fldusl34.png-61.9kB][47] ![image_1f9n8b1nk1lv114pg1o3h1hbpt1b3h.png-893.7kB][48] ``` 访问 Kuboard 在浏览器中打开链接 http://your-node-ip-address:30080 输入初始用户名和密码，并登录 用户名： admin 密码： Kuboard123 ``` ![image_1f9n8hpeu1b3m1k7v1vbhj3r1h4j4r.png-587.9kB][49] ![image_1f9n8im0u151hdaaikg1fnm1b5458.png-318.2kB][50] ![image_1f9n8jepj1mf216m79rj13vv193b65.png-493.2kB][51] ![image_1f9n8k56p1coc17mt1ohn1bu3n3j6i.png-669.4kB][52] ![image_1f9n8gaiu1587td1qstukp2nt3u.png-518.7kB][53] [1]: http://static.zybuluo.com/zhangyy/2duz4z80q4a2biot7c5qajxl/image_1f0sgr8qq1d2inqr35ml3g1qsu9.png [2]: http://static.zybuluo.com/zhangyy/ndw4mpaw4bcx8bcx9en1o4h2/image_1f1m9lqt31do01d3t1rgejtbfnn9.png [3]: http://static.zybuluo.com/zhangyy/8kzsmkiqgh9mk5wf3t2n29u3/image_1f0shlio9u63h1ct111vne154m.png [4]: http://static.zybuluo.com/zhangyy/4w2efw8xotok9v210hmvp3ve/image_1f0shp7p44mc1vit1cf1grd1i2s13.png [5]: http://static.zybuluo.com/zhangyy/v2irg31cgll48e7uuj2u3bf6/image_1f0sht5fipl21jlj9ck1ljlt8a1g.png [6]: http://static.zybuluo.com/zhangyy/x77qhtvkwmq92e8w4y4pqop4/image_1f1mj9sue1if77h9op982o48n16.png [7]: http://static.zybuluo.com/zhangyy/p7yvezincufexz2px90tz6jr/image_1f1mjbmf21bbjpftc3e1vqit7r1j.png [8]: http://static.zybuluo.com/zhangyy/ie0nqhyzulpreemzi661hadd/image_1f1mjfeet1qk215s96eg88h1oj820.png [9]: http://static.zybuluo.com/zhangyy/85wtjtjf797fyfjofiofeeq5/image_1f1mjhmfttl5m4k1a1s1ih5rkr2d.png [10]: http://static.zybuluo.com/zhangyy/eovypbjjf6qp20ftk5t3wqr7/image_1f1mk5gv41epj1kar2kq1ou41fsf2q.png [11]: http://static.zybuluo.com/zhangyy/ic9gud5xpoc9tahjlkqwtxmb/image_1f1mldn831cbj1g601idu7opqhv37.png [12]: http://static.zybuluo.com/zhangyy/03dtnl6hopsrmw73mdiizd3d/image_1f1mn8j171c4e1v8l1t051h1j1ibqm.png [13]: http://static.zybuluo.com/zhangyy/ewssjfuw0ouuypzbmv4mlyt9/image_1f1mn7ikttv911qe1nneifbgif9.png [14]: http://static.zybuluo.com/zhangyy/1cnplhx502e2ko89tge1eenx/image_1f1mq60mv7411bmk7nf1l6b1udr2j.png [15]: http://static.zybuluo.com/zhangyy/qhpcrdn84ffdtkesn4d72of8/image_1f1mqabg41t591dva1329rvn1j5q33.png [16]: http://static.zybuluo.com/zhangyy/eleebhesxkfdf5oymbm62523/image_1f1mqd2116eu1qa1jbp1819oqr3g.png [17]: http://static.zybuluo.com/zhangyy/7s7hlbfnq00knr8wnjh3b66z/image_1f1mqdlqh18smvcpg9dsf3mf63t.png [18]: http://static.zybuluo.com/zhangyy/t72feyel2k5rpb5h8kf7dxtj/image_1f1ms7272ufi33bcm1bit1b7e4q.png [19]: http://static.zybuluo.com/zhangyy/c7onayqvb1fiekjx8irgp7bi/image_1f1msjrle1p9817bd1u2etq0i1u57.png [20]: http://static.zybuluo.com/zhangyy/tuziindwxh8nwtgdh0lnq95p/image_1f1msni0f1vnbj3qvkeige1i6k5k.png [21]: http://static.zybuluo.com/zhangyy/qfe1fh2dgd46z8pequ3x4d4u/image_1f1msorv8stapetkm8nfm1ejp61.png [22]: http://static.zybuluo.com/zhangyy/4kztuuqyg3jd0t7rfzwesu36/image_1f1msplr71chm105p5k5bkfq0i6e.png [23]: http://static.zybuluo.com/zhangyy/efp1wb1n6mstjeie69x8qhz0/image_1f1msu790akc19nqrce57k1lnr6r.png [24]: http://static.zybuluo.com/zhangyy/7vj7t9xhylnvl6lim6zhi7i0/image_1f1mu5m9711nt11o4g6r1p9u1j7878.png [25]: http://static.zybuluo.com/zhangyy/k7364qqo76vkcuhcmqv21sx2/image_1f1mug5h37741ib913ruksd170g7l.png [26]: http://static.zybuluo.com/zhangyy/d5m2avxj8s5dlm52rnkmmjh2/image_1f1muher87hf4ebbs718ak17cp82.png [27]: http://static.zybuluo.com/zhangyy/87g5f5zemmad37re1r5djeb8/image_1f1muodoa1anri996om1eslnro8f.png [28]: http://static.zybuluo.com/zhangyy/zffrp2yn032ha6ezp0zcdsiy/image_1f1n1am3q1evr1lnh1eti1l0c166a8s.png [29]: http://static.zybuluo.com/zhangyy/1xqjckj3vn559si4o5vs8y6v/image_1f1n1bivn15lkptmvda15sk1f1999.png [30]: http://static.zybuluo.com/zhangyy/nd4wgi362fcw3i76eclvm0ns/image_1f9m4197m14vmmup1tfiede183d9.png [31]: http://static.zybuluo.com/zhangyy/f25z20nhs2gens7tqvurrckg/image_1f9m42fcr15681hpi1cmn7st1d59m.png [32]: http://static.zybuluo.com/zhangyy/07yvz5diulbjb3zp6qvvikdk/image_1f9m4bogl9g21v231tc16aev0e13.png [33]: http://static.zybuluo.com/zhangyy/quvf9suv22a0lgevul16ow62/image_1f9m4p2ed167q8r51vbr1n83lbj1g.png [34]: http://static.zybuluo.com/zhangyy/w7x3i6ptif1moqr1lh7rmrt4/image_1f9m4qfrlvign6k1venf2p12gj2d.png [35]: http://static.zybuluo.com/zhangyy/jasdgx3blcr6prxuv8t9rqog/image_1f9m4tjr7jnbd7p1rlk1lv911n5a.png [36]: http://static.zybuluo.com/zhangyy/m4v4wcnt4hssw6qp3oxnqkhp/image_1f9m4vn2219281hupojg8or14le5n.png [37]: http://static.zybuluo.com/zhangyy/2g8n80pjs46ykzhkx0yn8v2x/image_1f9m52b078d91ei3p0cu351eqp8k.png [38]: http://static.zybuluo.com/zhangyy/bmzr7y1zw1y715c2v2ycq2s1/image_1f9m64ouv1slimm81t5k1f5u15r391.png [39]: http://static.zybuluo.com/zhangyy/5cknm05l2ty7vsjbl3oqmcxq/image_1f9mf904t1grm1akf1nt51b1v1g879.png [40]: http://static.zybuluo.com/zhangyy/m6k1i9grhdwbix51ducv960l/image_1f9mflqut1moj1dqh9ojbko1pju26.png [41]: http://static.zybuluo.com/zhangyy/iw583xviklo4sembrdd3b9j9/image_1f9n6n32h1iko1st01r792si9fa9.png [42]: http://static.zybuluo.com/zhangyy/em4xkis4ech3b9foxs6ovgf8/image_1f9n6tps337kp864141oq21v9p13.png [43]: http://static.zybuluo.com/zhangyy/ld3q0967mx1h2xt1nixkwbrx/image_1f9n72ituq841b7nfrkare1jlp1g.png [44]: http://static.zybuluo.com/zhangyy/mucmmf7hnv1y6aqrk5xln68l/image_1f9n73hqf1a24qkjsf37ms12et1t.png [45]: http://static.zybuluo.com/zhangyy/wmxdrv9q1gcjdpcctob84k6o/image_1f9n74aga151hkhfknnhlts3n2a.png [46]: http://static.zybuluo.com/zhangyy/kld9zgho2yfx416zyv5tdr3d/image_1f9n87u3i12901tq0ad71ku71er52n.png [47]: http://static.zybuluo.com/zhangyy/49ys189fjqops9agwitc2mg7/image_1f9n89h5e1sbs4h41pf7fldusl34.png [48]: http://static.zybuluo.com/zhangyy/43cdswcagpokcybzfwndrfke/image_1f9n8b1nk1lv114pg1o3h1hbpt1b3h.png [49]: http://static.zybuluo.com/zhangyy/myfn98938pj7pzg8yehu8e5j/image_1f9n8hpeu1b3m1k7v1vbhj3r1h4j4r.png [50]: http://static.zybuluo.com/zhangyy/9yqlpgzz6fr8bj6qkobdyzrg/image_1f9n8im0u151hdaaikg1fnm1b5458.png [51]: http://static.zybuluo.com/zhangyy/ac3yskex6aex9oisanfo0h1r/image_1f9n8jepj1mf216m79rj13vv193b65.png [52]: http://static.zybuluo.com/zhangyy/lthwo52hmo42xpv41ocos4nj/image_1f9n8k56p1coc17mt1ohn1bu3n3j6i.png [53]: http://static.zybuluo.com/zhangyy/zkjzzp64pfbiyha8i2tzuiqz/image_1f9n8gaiu1587td1qstukp2nt3u.png