基于Ubuntu16.04和kubeadm部署kuernetes

1.部署环境：

     2台虚拟机，Ubuntu16.04系统。设定A虚拟机的IP 为A-IP，B虚拟机的IP为B-IP，并A虚拟机作为master。

2. A、B两台虚拟机都需要做的部署操作：

     建议以root用户执行下列操作。

     a. 跟新系统apt包：

 apt-get update -y apt-get upgrade -y

    b. 关闭开启Swap的设备，kubernetes是建议关闭的：   

 Swapoff -a

        上述命令，可以临时关闭，机器重启之后需要重新设置。

        具体的swap扩展信息，可参考http://www.runoob.com/linux/linux-comm-swapoff.html

     c. 安装Docker，kubernetes需要机遇Docker环境运行：

 apt-get install apt-transport-https -y curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - apt-get install docker.io -y

         查看docker版本及启动docker service:

docker version ###### Client: Version: 17.03.2-ce API version: 1.27 Go version: go1.6.2 Git commit: f5ec1e2 Built: Thu Jul 5 23:07:48 2018 OS/Arch: linux/amd64 Server: Version: 17.03.2-ce API version: 1.27 (minimum version 1.12) Go version: go1.6.2 Git commit: f5ec1e2 Built: Thu Jul 5 23:07:48 2018 OS/Arch: linux/amd64 Experimental: false ###### systemctl enable docker systemctl start docker systemctl status docker ###### docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2018-11-24 06:58:36 EST; 3min 57s ago Docs: https://docs.docker.com Main PID: 14473 (dockerd) CGroup: /system.slice/docker.service ├─14473 /usr/bin/dockerd -H fd:// └─14483 docker-containerd -l unix:///var/r............. ######

         如果docker service的状态为active(running)，则说明docker 安装并启动成功。

    d. 安装kubernetes：

sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - echo 'deb http://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list apt-get update apt-get install -y kubelet kubeadm kubectl kubernetes-cni

3. 配置master节点：

sudo kubeadm init --pod-network-cidr 10.244.0.0/16

         --pod-network-cidr是指配置节点中的pod的可用IP地址，此为内部IP

        初始化kubeadm成功之后，会输出如下信息：

Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join A-IP:6443 --token y89hw4.jahkc29856dc6pi6 --discovery-token-ca-cert-hash sha256:3cddcf79c0d2ffa338167589080976876978f3ede9a2014f5aa4f5104a

         执行如下命令，配置master网络：

sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml

        在其他node，执行如下命令，可将node加入到master。该命令中的参数来自于上一个输出信息：

kubeadm join A-IP:6443 --token y89hw4.jahkc29856dc6pi6 --discovery-token-ca-cert-hash sha256:3cddcf79c0d2ffa338167589080976876978f3ede9a2014f5aa4f5104a

      执行如下命令去配置kubectl:

mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config

       至此，执行如下命令即可获取到node的相关信息：

kubectl get nodes NAME STATUS ROLES AGE VERSION kubernetes-master NotReady master 14m v1.12.2

4. 添加其他节点(node)到master:

kubeadm join A-IP:6443 --token y89hw4.jahkc29856dc6pi6 --discovery-token-ca-cert-hash sha256:3cddcf79c0d2ffa338167589080976876978f3ede9a2014f5aa4f5104a ###### [discovery] Trying to connect to API Server "A-IP:6443" [discovery] Created cluster-info discovery client, requesting info from "https://B-IP:6443" [discovery] Requesting info from "https://B-IP:6443" again to validate TLS against the pinned public key [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "B-IP:6443" [discovery] Successfully established connection with API Server "A-IP:6443" This node has joined the cluster: * Certificate signing request was sent to master and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the master to see this node join the cluster. ######

        在master节点执行 kubectl get nodes，可以检查是否成功加入。

5. 配置kubernetes-dashboard:

     本次测试是安装kubernetes-dashboard，并使得能够远程访问。

     执行如下信息，安装kubernetes-dashboard：

apiVersion: v1 kind: List items: - apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kube-system type: Opaque - apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system - kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: kubernetes-dashboard-minimal namespace: kube-system rules: # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret. - apiGroups: [""] resources: ["secrets"] verbs: ["create"] # Allow Dashboard to create 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] verbs: ["create"] # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics from heapster. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:"] verbs: ["get"] - apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard-minimal namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubernetes-dashboard-minimal subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system - kind: Deployment apiVersion: apps/v1beta2 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: - name: kubernetes-dashboard image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.0 ports: - containerPort: 8443 protocol: TCP args: - --auto-generate-certificates # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. # - --apiserver-host=http://my-address:port volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs # Create on-disk volume to store exec logs - mountPath: /tmp name: tmp-volume livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule - kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: type: NodePort ports: - port: 443 targetPort: 8443 nodePort: 30000 selector: k8s-app: kubernetes-dashboard 

         上述配置文件，service中开启nodeport，以便于远程能够直接访问。

        文件执行完成后，可以通过https://A-IP:30000 来访问。

        在kubernetes 1.7之后建议使用token去登录：

        执行如下命令获取token：

kubectl -n kube-system get secret ####### attachdetach-controller-token-pqtcp kubernetes.io/service-account-token 3 3h18m bootstrap-signer-token-qnx4s kubernetes.io/service-account-token 3 3h18m bootstrap-token-y89hw4 bootstrap.kubernetes.io/token 7 3h18m certificate-controller-token-r6wl6 kubernetes.io/service-account-token 3 3h18m clusterrole-aggregation-controller-token-6mrrv kubernetes.io/service-account-token 3 3h18m coredns-token-f4nfd kubernetes.io/service-account-token 3 3h18m cronjob-controller-token-thn79 kubernetes.io/service-account-token 3 3h18m daemon-set-controller-token-mwszv kubernetes.io/service-account-token 3 3h18m default-token-xbxx9 kubernetes.io/service-account-token 3 3h18m deployment-controller-token-dkj7n kubernetes.io/service-account-token 3 3h18m disruption-controller-token-nnq57 kubernetes.io/service-account-token 3 3h18m endpoint-controller-token-6l95q kubernetes.io/service-account-token 3 3h18m expand-controller-token-qxb7h kubernetes.io/service-account-token 3 3h18m flannel-token-ttkmd kubernetes.io/service-account-token 3 3h2m generic-garbage-collector-token-bjndg kubernetes.io/service-account-token 3 3h18m horizontal-pod-autoscaler-token-kpds7 kubernetes.io/service-account-token 3 3h18m job-controller-token-9hh7b kubernetes.io/service-account-token 3 3h18m kube-proxy-token-wdv62 kubernetes.io/service-account-token 3 3h18m kubernetes-dashboard-certs Opaque 0 171m kubernetes-dashboard-key-holder Opaque 2 171m kubernetes-dashboard-token-m97pg kubernetes.io/service-account-token 3 171m namespace-controller-token-cqmtt kubernetes.io/service-account-token 3 3h18m node-controller-token-nx6cv kubernetes.io/service-account-token 3 3h18m persistent-volume-binder-token-55gtw kubernetes.io/service-account-token 3 3h18m pod-garbage-collector-token-d26sh kubernetes.io/service-account-token 3 3h18m pv-protection-controller-token-dj6mt kubernetes.io/service-account-token 3 3h18m pvc-protection-controller-token-98wbx kubernetes.io/service-account-token 3 3h18m replicaset-controller-token-kfnq4 kubernetes.io/service-account-token 3 3h18m replication-controller-token-wqpph kubernetes.io/service-account-token 3 3h18m resourcequota-controller-token-rzwsc kubernetes.io/service-account-token 3 3h18m service-account-controller-token-kmsfr kubernetes.io/service-account-token 3 3h18m service-controller-token-k9ps4 kubernetes.io/service-account-token 3 3h18m statefulset-controller-token-j4t45 kubernetes.io/service-account-token 3 3h18m token-cleaner-token-qcbnj kubernetes.io/service-account-token 3 3h18m ttl-controller-token-l9bqk kubernetes.io/service-account-token 3 3h18m ####### kubectl -n kube-system describe secret clusterrole-aggregation-controller-token-6mrrv ####### Name: clusterrole-aggregation-controller-token-6fzrv Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: clusterrole-aggregation-controller kubernetes.io/service-account.uid: e132b88c-efe2-11e8-b652-005056a0b094 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1025 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjbHVzdGVycm9sZS1hZ2dyZWdhdGlvbi1jb250cm9sbGVyLXRva2VuLTZmenJ2Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNsdXN0ZXJyb2xlLsdnfksdnfksdnfkjfieekjfskdjfsdxsZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJlMTMyYjg4Yy1lZmUyLTExZTgtYjY1Mi0wMDUwNTZhMGIwOTQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06Y2x1c3RlcnJvbGUtYWdncmVnYXRpb24tY29udHJvbGxlciJ9.cLhchbNMkyPTLVqk0MuWP6_7yC1zjJ_MW3R0Tv_uyEtFuYNGcYXKVwXilO2TtMzp5aeqcHq1D-tT21_a0RzJZzd4OktIANed1Ix5PRAHVRX8o7O2jK2Dj9_neqZOogPblKz7jR6g82DQf8R_Vphq4MP3CetEeUIUxbY897r1xKpb7oizZg_ca-Ai7qnQgeCw6ii7O2s1UA8ugYldaXwv7lUKze3bCCSFX9o8dUlvy8WtgF3wVvb7WW3mId1t4nGg9oXlzxhSG9z5z-BtvWNXIp1dVhrFNQgEY9hiipxgpoagLw45FxaOvtfbWd7Mhlrxotf3iED60P6e7FbS4Z_Dgw #######

        其中clusterrole-aggregation-controller-token-6mrrv所在的pod中存在kubernetes-dashboard登录所需的token。