首页 文章 精选 留言 我的

精选列表

搜索[ATH事业群]，共10000篇文章
优秀的个人博客,低调大师

2015-03-22 网易笔试（数据挖掘方向）——邮件事业部

答案正在更新，有想法的也可以留言............ 一：单选题 1：下列程序的输出结果为（） #include <iostream.h> void main() { int n[][] = {10,20,30,40,50,60}; int (*p)[3]; p = n; cout<<[0][0] << "," <<*( p[0] + 1) << "," <<(*p)[2]<<endl; } A: 10,30,50 B: 10,20,30 C: 20,40,60 D: 10,30,60 解析： n[2][3] = { 10,20,30, 40,50,60 }; *( p[0] + 1) = p[i][j] (与此类似的形式还有 *( *( p+i ) + j )) 故等于20 (*p)[2]:*p指的是首行 2代表第三列 所以为 30 答案选B 2：存储以下数据，占用字节最多的是（） A: 0 B: '0' C: " 0 " D: 0.0 解：int 在不同位数的计算机上表现出的长度不一样，其长度至少为2字节（在16位的计算机上），在32位的计算机上其长度为4字节，64位的计算机上长度为8字节 char 字段字节长度为8 “0”的长度为2 0.0的长度为8 3: 栈和队列共同的特点是 A: 只允许在端点处插入和删除 B: 都是先进后出 C: 都是先进先出 D: 没有共同点 解：栈和队列都是数据结构中的一种，栈是先进后出，其删除和插入数据只能在端点处进行，对列是先进先出型，插入和删除数据也只能在端点处进行，另外一个比较混淆的点是 栈通常用于深度遍历， 而队列用于广度遍历。 4: 任何一颗二叉树的叶节点在前序，中序，后序遍历序列中的相对次序 A: A和B 右方 B: A 和B祖先 C: A 和B左方 D: A 是B子孙 解：这道题题目的意思给的太迷乱了，所以我的理解也不一定正确，我的理解是A和B是两个叶子节点，共有一个父结点，所以无论是前序，中序还是后序遍历中，A总是在B的左边，所以选择C 5：下列关于MapReduce的说法正确的是？ A: MapReduce有多个输入路径时，文件类型必须保持一致 B: 可以使用Counter观察MR Job运行的各种细节数据 C; 使用TextInputFormat时，Mapper的key类型为Text，value类型为LongWritable D: 以上都正确 解：MapReduce有多个输入路径时不需要保证文件类型一致，可以使用MultipleInouts类来指定多个Mapper函数，只要一个Map函数来处理一种类型的文件即可。 MapReduce Counter为提供我们一个窗口：观察MapReduce job运行期的各种细节数据 使用InputFormat时Mapper的key为LongWritable，表示偏移量，Text表示一行的文本 故选择 B 6：PCA降维中用到的算法是： A: 牛顿法 B: sgd C: svd D: Cutting-Plane 解：牛顿法：迭代算法 sgd：梯度下降算法 svd：推荐算法（Singular Value Decomposition） Cutting-Plane：列生成算法 其实小编对这几个算法并不大懂，所以具体的还望大神指导，具体答案小编目测是选B 7： A: 0.59 0.514 B: 1.14 1.02 C: 0.59 1.02 D; 0.514 0.59 解：B 8：请选出用于无监督学习问题的方法 A: SGD B: 反向传播 C: BFGS D: 幂迭代 解：D SGD：梯度下降算法 反向传播：Backpropagation algorithm，简称：BP算法，是一种监督学习算法，常被用来训练多层感知机 BFGS拟牛顿法，详情参考 幂迭代：聚类算法 9：Python中的浅拷贝会创建一个新的对象，但他包含的是对原始对象中包含项的引用，下列哪个操作不涉及浅拷贝 A: copy() B: list() C: 完全切片方法 D: 赋值（=） 解：答案选D 推荐一篇便于理解浅拷贝和深拷贝的文章：点击阅读 思路一：利用切片操作和工厂方法list方法拷贝就叫浅拷贝，只是拷贝最外围的对象本身，内部的元素都只是拷贝了一个引用而已。 思路二：利用copy中的deepcopy方法进行拷贝就叫做深拷贝，外围和内部元素都进行了拷贝对象本身，而不是引用。 但是对于数字，字符串和其他原子类型对象等，没有被拷贝的说法，即便是用深拷贝，查看id的话也是一样的，如果对其重新赋值，也只是新创建一个对象，替换掉旧的而已。 10：有一个文件ip.txt ，每行一条ip记录，共若干行，下列哪个命令可以实现统计出现次数最多的前三个ip及其次数 A: uniq -c ip.txt | sort -rn | head -n 3 B: sort ip.txt | uniq -c | sort -rn | head -n 3 C: cat ip.txt | count -n | sort -rn | head -n 3 D: cat ip.txt | sort | uniq -c | sort -rn | top -n 3 解：B linux基本命令具体自行百度 二：多选题 11：ls -l 显示如下 -rwxrw-r-- 1 aaa bbb 0 3月 4 11:21 ccc 下列那些说法是正确的/ A: 该文件是个目录 B: 该文件拥有者所在的组用户有权限修改文件 C: 该文件的权限数字表示为764 D: 该文件的拥有者是bbb 解：BC 参考 12：下列属于线性分类器的是 A: 决策树 B; 带和核函数的SVM C: logistics回归 D: 感知器 解：BCD 线性分类器：单层感知器网络、贝叶斯 非线性分类器：多层感知器网络、决策树 SVM本身是线性分类器，带核函数之后相当于把低维空间的数据映射到高维空间，之后就可以用线性分类器进行分类了 13： 下列常用的机器学习模型中那些属于生成模型 A: 朴素贝叶斯 B: 隐马尔可夫 C: 感知机 D: 决策树 解：B 朴素贝叶斯和隐马尔可夫是生成模型 感知机，决策树是判别模型 关于生成模型和判别模型结束 14：下列说法正确的是 A:StringBuilder是线程不安全的 B: java类可以同时用abstract 和 final声明 C: HasnMap中，使用get（key）==null 可以判断这个hashmap是否包含这个key D: volatike关键字不保证对变量操作的原子性 解：A，C，D abstract类是抽象类，必须做父类 15：Python中怎样替换一个字符串 A: String模块的sub函数 B: re模块的replace()函数 C: re模块的subn()函数 D: re模块的sub()函数 解：B,C,D A中string模块meiyousub方法，有replace函数 16：Spark中的RDD的持久化操作，下面说法正确的是？ A: 调用persist()之后不需要主打unpersist() B: MEMORY_ONLY_SEP模式比MEMORY_ONLY模式快 C: OFF_HEAP模式不需要序列化数据 D: MEMORY_ADD_DISK模式当内存放马不下数据时会将部分数据存储到磁盘中 解：B，C，D 楼主不懂Spark，是百度之后的答案，仅供参考 17：对154个元素组成的有序表进行二分法查找，可能的比较次数为： A: 10 B: 8 C: 4 D: 1 解：画一个二叉树共八层，所以小于等于8即可 B,C,D 18：有一棵二叉树的前序遍历和后序遍历分别是1,2,3,4和4,3,2,1,则该二叉树的中序遍历可能是 A: 1,2,3,4 B: 2,3,4,1 C: 3,2,4,1 D: 4,3,2,1 解：C,D 19：已知logistic回归分类器的分类阀值是0.5,训练集中正负类比例为3：1,预测性能中正类的precision为0.88 recall为0.81 ，以下那些处理手段可能提高预测结果中正类的recall A: 降低分类判断的阀值为0.3 B: 训练集中正负类的训练比例调整为6：1 C: 加大logistic回归的正则项的权重值 D: 降低logistic回归的正则项的权重值 解： 20：以下随机变量的概率图模型，那些图是满足p( c,b | a) = p(c | a)p(b |a ) A: B: C: D: 解：A,C（楼主不太懂的路过,,,,,） 三：主观题 21： 解： 22： 解： 23：请描述Java的Thread类中的start（）和run（）两个方法的区别？ 解： 24：（1）在SVM模型中，核函数的作用是什么？存在哪几种核函数 （2）对于SVM模型来说，松弛变量有何作用？ 25：现在有两种类型的文字新闻：体育新闻和财经新闻各10000条（已分好类），想以此作为训练样本实现一个这两种新闻的自动分类器，请描述如何实现（从如何将一条新闻转化为一条用于训练的特征向量） 解：
2016-03-22
优秀的个人博客,低调大师

Apache ZooKeeper 初认识（附用户交流群）

4.1 初识 ZooKeeper 在本节中，我们会对 ZooKeeper 进行一个初步的介绍，从 ZooKeeper 是什么、ZooKeeper 的由来及其基本概念展开，同时会向读者介绍使用ZooKeeper 来解决分布式一致性问题的优势。 4.1.1 ZooKeeper介绍 ZooKeeper是一个开放源代码的分布式协调服务，由知名互联网公司雅虎创建，是Google Chubby的开源实现。ZooKeeper的设计目标是将那些复杂且容易出错的分布式一致性服务封装起来，构成一个高效可靠的原语集，并以一系列简单易用的接口提供给用户使用。 ZooKeeper是什么ZooKeeper是一个典型的分布式数据一致性的解决方案，分布式应用程序可以基于它实现诸如数据发布/订阅、负载均衡、命名服务、分布式协调/通知、集群管理、Master选举、分布式
2019-05-20
优秀的个人博客,低调大师

云主机搭建Kubernetes 1.10集群

一、基础环境 云主机 下载软件包 将所有软件下载至/data目录 # 链接：https://pan.baidu.com/s/13DlR1akNBCjib5VFaIjGTQ 密码：1l69 # 链接：https://pan.baidu.com/s/1V6Uuj6a08mEq-mRLaI1xgw 密码：6gap master到node做免密认证 ssh-keygen ssh-copy-id root@192.168.1.237 ssh-copy-id root@192.168.1.100 ssh-copy-id root@192.168.1.188 设定主机名与host文件 # 分别设定node与master的主机名 hostnamectl set-hostname master exec bash # 同步所有主机的hosts文件 vim /etc/hosts 192.168.1.78 master localhost 192.168.1.237 node1 192.168.1.100 node2 192.168.1.188 node3 解决DNS解析localhost 此云主机的DNS解析localhost会解析到一个鬼地址，这是个大坑。kubeadm初始化是会用到localhost。如果你的主机能解析到自己的IP，那么这步可以跳过。如果不能则需要自己搭建一个DNS，将localhost解析到自己。 # 1.检测 [root@node2 ~]# nslookup localhost Server: 118.118.118.9 Address: 118.118.118.9#53 Non-authoritative answer: Name: localhost.openstacklocal Address: 183.136.168.91 # 2.搭建DNS yum -y install dnsmasq cp /etc/resolv.conf{,.bak} rm -rf /etc/resolv.conf echo -e "nameserver 127.0.0.1\nnameserver $(hostname -i)" >> /etc/resolv.conf chmod 444 /etc/resolv.conf chattr +i /etc/resolv.conf echo -e "server=8.8.8.8\nserver=8.8.4.4" > /etc/dnsmasq.conf echo -e "$(hostname -i)\tlocalhost.$(hostname -d)" >> /etc/hosts service dnsmasq restart # 3.再次检测 [root@master ~]# nslookup localhost Server: 127.0.0.1 Address: 127.0.0.1#53 Name: localhost Address: 192.168.1.78 同步系统时间 ntpdate 0.centos.pool.ntp.org 关闭防火墙 iptables -F systemctl stop firewalld systemctl disable firewalld 关闭SELinux & 关闭swap swapoff -a sed -i 's/.*swap.*/#&/' /etc/fstab setenforce 0 确认时区 timedatectl set-timezone Asia/Shanghai systemctl restart chronyd.service 修改系统参数 cat <<EOF > /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system 安装docker tar -xvf docker-packages.tar cd docker-packages yum -y install local *.rpm systemctl start docker && systemctl enable docker 配置镜像加速器 vim /etc/docker/daemon.json { "registry-mirrors": ["https://lw9sjwma.mirror.aliyuncs.com"] } systemctl daemon-reload systemctl restart docker 配置k8s的yum源 vim /etc/yum.repos.d/k8s.repo [k8s] name=k8s baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ gpgcheck=0 获取kube软件包 cd kube-packages-1.10.1 # 软件包在网盘中下载 tar -xvf kube-packages-1.10.1.tar cd kube-packages-1.10.1 yum -y install local *.rpm systemctl start kubelet && systemctl enable kubelet 统一k8s与docker的驱动 # 1.查看docker驱动 docker info | Cgroup Driver Cgroup Driver: cgroupfs # 修改k8s配置文件与docker保持一致 sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf 导入基础镜像 cd /data docker load -i k8s-images-1.10.tar.gz 二、初始化master节点 # 初始化master 指定的版本要与kubeadm版本一致 # kubeadm只给定了最少选项，集群名称等等都没有指定，kubeadm init [root@master ~]# kubeadm init --kubernetes-version=v1.10.1 --pod-network-cidr=10.244.0.0/16 # 初始化完成后得到如下信息 Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.1.78:6443 --token qabol0.c2gq0uyfxvpqr8bu --discovery-token-ca-cert-hash sha256:2237ec7b8efd5a8f68adcb04900a0b17b9df2a78675a7d62b4aef644a7f62c05 # kubeadm join 是node节点加入集群的命令，注意token的有效期 若要通过普通用户使用群集，需要执行以下步骤 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config 基本命令 # 查看pods kubectl get pods # 查看系统pods [root@master ~]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE etcd-master 0/1 Pending 0 1s kube-apiserver-master 0/1 Pending 0 1s kube-controller-manager-master 0/1 Pending 0 1s kube-dns-86f4d74b45-d42zm 0/3 Pending 0 8h kube-proxy-884h6 1/1 NodeLost 0 8h kube-scheduler-master 0/1 Pending 0 1s # 查看集群各组件状态信息 [root@master ~]# kubectl get componentstatuses NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {"health": "true"} You have new mail in /var/spool/mail/root 三、node加入集群 # 确保node节点cgroup驱动保持一致 sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf # 命令来自集群初始化之后额显示中 kubeadm join 192.168.1.78:6443 --token v0866r.u7kvg5js1ah2u1bi --discovery-token-ca-cert-hash sha256:7b36794f4fa5121f6a5e309d0e312ded72997a88236a93ec7da3520e5aaccf0e # master节点查看nodes信息 [root@master data]# kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady master 57m v1.10.1 node1 NotReady <none> 27m v1.10.1 node2 NotReady <none> 11s v1.10.1 node3 NotReady <none> 4s v1.10.1 You have new mail in /var/spool/mail/root 四、部署网络 部署 flannel官网 flannel下载时不用科学上网,flannel的yml文件会自动去quay.io网站中下载镜像。 # 1.1使用软件包中的flannel,并指pod映射到哪个主机的网卡上面。 vim kube-flannel.yml command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr","-iface=eth0" ] # 以下要按顺序创建，先创建rbac，之前没有穿件rbac导致pod正常创建，但是pin不同 kubectl apply -f kube-flannel-rbac.yml kubectl apply -f kube-flannel.yml # 后，节点的状态会变为ready [root@master1 kubernetes1.10]# kubectl get node NAME STATUS ROLES AGE VERSION master Ready master 57m v1.10.1 node1 Ready <none> 27m v1.10.1 node2 Ready <none> 11s v1.10.1 node3 Ready <none> 4s v1.10.1 # 2.从官网下载最新的flannel，k8s1.7+ 直接执行以下命令即可 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml flannel配置文件修改 kube-flannel.yml中指定使用的网段 "Network": "10.244.0.0/16" 默认使用16位掩码，则在各node中都分配一个10.244.0.0/8的网络 五、部署dashboard kubectl apply -f kubernetes-dashboard-http.yam kubectl apply -f admin-role.yaml kubectl apply -f kubernetes-dashboard-admin.rbac.yaml 命令行常用命令 # 查看pod信息，默认显示default名称空间下的pod [root@master ~]# kubectl get pods No resources found. # 指定名称空间写pod [root@master ~]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE etcd-master 1/1 Running 0 3h kube-apiserver-master 1/1 Running 0 3h kube-controller-manager-master 1/1 Running 0 3h kube-dns-86f4d74b45-bzbvc 3/3 Running 0 3h kube-flannel-ds-5ghhj 1/1 Running 0 2h kube-flannel-ds-ht4xd 1/1 Running 0 3h kube-flannel-ds-kbm5g 1/1 Running 0 3h kube-flannel-ds-mlj4r 1/1 Running 0 2h kube-proxy-9xxnd 1/1 Running 0 3h kube-proxy-n9w5x 1/1 Running 0 3h kube-proxy-nkn8c 1/1 Running 0 2h kube-proxy-shd6l 1/1 Running 0 2h kube-scheduler-master 1/1 Running 0 3h kubernetes-dashboard-5c469b58b8-rjfx6 1/1 Running 0 1h # 显示更详细的pod信息，此时各pod中都运行了一个kube-proxy和flannel容器 -o wide 显示更详细的信息，报错node节点iP、主机名 [root@master ~]# kubectl get pods -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE etcd-master 1/1 Running 0 3h 192.168.1.78 master kube-apiserver-master 1/1 Running 0 3h 192.168.1.78 master kube-controller-manager-master 1/1 Running 0 3h 192.168.1.78 master kube-dns-86f4d74b45-bzbvc 3/3 Running 0 3h 10.244.0.2 master kube-flannel-ds-5ghhj 1/1 Running 0 2h 192.168.1.188 node3 kube-flannel-ds-ht4xd 1/1 Running 0 3h 192.168.1.78 master kube-flannel-ds-kbm5g 1/1 Running 0 3h 192.168.1.237 node1 kube-flannel-ds-mlj4r 1/1 Running 0 2h 192.168.1.100 node2 kube-proxy-9xxnd 1/1 Running 0 3h 192.168.1.237 node1 kube-proxy-n9w5x 1/1 Running 0 3h 192.168.1.78 master kube-proxy-nkn8c 1/1 Running 0 2h 192.168.1.100 node2 kube-proxy-shd6l 1/1 Running 0 2h 192.168.1.188 node3 kube-scheduler-master 1/1 Running 0 3h 192.168.1.78 master kubernetes-dashboard-5c469b58b8-rjfx6 1/1 Running 0 1h 10.244.0.3 master 六、kubeadm清空配置 # 清空kubectl kubeadm reset # 清空网络信息 ip link del cni0 ip link del flannel.1 七、踩过的那些坑 确保master与node的DNS解析localhost能解析到自己的IP node加入master确保token不过期 node确保kubelet正常启动并运行 flannel网络要先创建kube-flannel-rbac.ymal再创建 kube-flannel.yml 八、token过期的解决办法 # 1.查看已经存在的token kubeadm token list # 2.创建token kubeadm token create # 3.查看ca证书的sha256编码 openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' # 4.node使用新的token加入集群 kubeadm join --token acb123 --discovery-token-ca-cert-hash sha256:efg456 172.16.6.79:6443 --skip-preflight-checks # abc123 新创建的Token # efg456 证书的sha256编码 # IP+Port Master的IP+Port 感谢 无痴迷，不成功 discsthnew
2018-07-08
优秀的个人博客,低调大师

hadoop2集群搭建过程

在这里我选用4台机器进行示范，各台机器的职责如下表格所示 （说明： 1. ZooKeepe使用其它节点的 2. Hadoop0节点安装所有的master, ha的matster全部是worker, 以达到较高的资源利用率，又对master的负载不会过高） Hadoop0 -> dchadoop206 Hadoop1 -> dchadoop207 Hadoop2 -> dchadoop208 Hadoop3 -> dchadoop209 1.搭建自动HA 1.1.复制编译后的hadoop项目到/usr/local目录下 1.2.修改位于etc/hadoop目录下的配置文件 1.1.1.hadoop-env.sh 1 export JAVA_HOME= /usr/local/jdk 1.1.1.core-site.xml 1 2 3 4 5 <configuration> <property> <name>fs.defaultFS< /name > <value>hdfs: //cluster1 < /value > < /property > 【这里的值指的是默认的HDFS路径。当有多个HDFS集群同时工作时，用户如果不写集群名称，那么默认使用哪个哪？在这里指定！该值来自于hdfs-site.xml中的配置。在节点hadoop0和hadoop1中使用cluster1，在节点hadoop2和hadoop3中使用cluster2】 1 2 3 4 <property> <name>hadoop.tmp. dir < /name > <value> /data0/hadoop/tmp < /value > < /property > 【这里的路径默认是NameNode、DataNode、JournalNode等存放数据的公共目录。用户也可以自己单独指定这三类节点的目录。】 1 2 3 4 <property> <name>ha.zookeeper.quorum< /name > <value>hadoop0:2181,hadoop1:2181,hadoop2:2181< /value > < /property > 【这里是ZooKeeper集群的地址和端口。注意，数量一定是奇数，且不少于三个节点】 1 < /configuration > 1.1.1.hdfs-site.xml 该文件只配置在hadoop0和hadoop1上。 1 2 3 4 5 <configuration> <property> <name>dfs.replication< /name > <value>2< /value > < /property > 【指定DataNode存储block的副本数量。默认值是3个，我们现在有4个DataNode，该值不大于4即可。】 1 2 3 4 <property> <name>dfs.namenode.name. dir < /name > <value> file : ///data0/hadoop2/hdfs/name < /value > < /property > 【指定namenode元数据信息存储位置】 1 2 3 <property> <name>dfs.datanode.data. dir < /name ><value> file : ///data0/hadoop2/hdfs/data , file : ///data1/hadoop2/hdfs/data , file : ///data2/hadoop2/hdfs/data , file : ///data3/hadoop2/hdfs/data , file : ///data4/hadoop2/hdfs/data , file : ///data5/hadoop2/hdfs/data , file : ///data6/hadoop2/hdfs/data , file : ///data7/hadoop2/hdfs/data , file : ///data8/hadoop2/hdfs/data , file : ///data9/hadoop2/hdfs/data , file : ///data10/hadoop2/hdfs/data < /value > < /property > 【指定datanode元数据信息存储位置,设置成所有的磁盘】 1 2 3 4 <property> <name>dfs.nameservices< /name > <value>cluster1< /value > < /property > 【设置cluster1的namenode id。】 1 2 3 4 <property> <name>dfs.ha.namenodes.cluster1< /name > <value>hadoop0,hadoop1< /value > < /property > 【指定NameService是cluster1时的namenode有哪些，这里的值也是逻辑名称，名字随便起，相互不重复即可】 1 2 3 4 <property> <name>dfs.namenode.rpc-address.cluster1.hadoop0< /name > <value>hadoop0:9000< /value > < /property > 【指定hadoop0的RPC地址】 1 2 3 4 <property> <name>dfs.namenode.http-address.cluster1.hadoop0< /name > <value>hadoop0:50070< /value > < /property > 【指定hadoop0的http地址】 1 2 3 4 <property> <name>dfs.namenode.rpc-address.cluster1.hadoop1< /name > <value>hadoop1:9000< /value > < /property > 【指定hadoop1的RPC地址】 1 2 3 4 <property> <name>dfs.namenode.http-address.cluster1.hadoop1< /name > <value>hadoop1:50070< /value > < /property > 【指定hadoop1的http地址】 1 2 3 4 <property> <name>dfs.namenode.shared.edits. dir < /name > <value>qjournal: //hadoop0 :8485;hadoop1:8485;hadoop2:8485 /cluster1 < /value > < /property > 【指定cluster1的两个NameNode共享edits文件目录时，使用的JournalNode集群信息】 1 2 3 4 <property> <name>dfs.ha.automatic-failover.enabled.cluster1< /name > <value> true < /value > < /property > 【指定cluster1是否启动自动故障恢复，即当NameNode出故障时，是否自动切换到另一台NameNode】 1 2 3 4 <property> <name>dfs.client.failover.proxy.provider.cluster1< /name > <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider< /value > < /property > 【指定cluster1出故障时，哪个实现类负责执行故障切换】 1 2 3 4 <property> <name>dfs.journalnode.edits. dir < /name > <value> /data0/hadoop2/hdfs/journal < /value > < /property > 【指定JournalNode集群在对NameNode的目录进行共享时，自己存储数据的磁盘路径】 1 2 3 4 <property> <name>dfs.ha.fencing.methods< /name > <value>sshfence< /value > < /property > 【一旦需要NameNode切换，使用ssh方式进行操作】 1 2 3 4 <property> <name>dfs.ha.fencing. ssh .private-key-files< /name > <value> /root/ . ssh /id_rsa < /value > < /property > 【如果使用ssh进行故障切换，使用ssh通信时用的密钥存储的位置】 1 < /configuration > 1.2.4. slaves hadoop1 hadoop2 hadoop2 1.3.把以上配置的内容复制到hadoop1、hadoop2、hadoop3节点上 1.4.修改hadoop1、hadoop2、hadoop3上的配置文件内容 1.4.1.修改hadoop2上的core-site.xml内容 1 fs.defaultFS的值改为hdfs: //cluster2 1.4.1.修改hadoop2上的hdfs-site.xml内容 把cluster1中关于journalnode的配置项删除，增加如下内容 1 2 3 4 <property> <name>dfs.namenode.shared.edits. dir < /name > <value>qjournal: //hadoop0 :8485;hadoop1:8485;hadoop2:8485 /cluster2 < /value > < /property > 1.4.3.开始启动 1.4.3.1.启动journalnode 在hadoop0、hadoop1、hadoop2上执行 1 sbin /hadoop-daemon .shstartjournalnode 1.4.3.1.格式化ZooKeeper 在hadoop0、hadoop2上执行 bin/hdfszkfc -formatZK zkCli.sh-->ls->/Hadoop-ha/cluster1 1.4.3.3.对hadoop0节点进行格式化和启动 1 bin /hdfs namenode- format 1 sbin /hadoop-daemon .shstartnamenode 1.4.3.4.对hadoop1节点进行格式化和启动 1 bin /hdfs namenode-bootstrapStandby 1 sbin /hadoop-daemon .shstartnamenode 1.4.3.5.在hadoop0、hadoop1上启动zkfc 1 sbin /hadoop-daemon .shstartzkfc 我们的hadoop0、hadoop1有一个节点就会变为active状态。 1.4.3.6.对于cluster2执行类似操作 1.4.4.启动datanode 在hadoop0上执行命令 1 sbin /hadoop-daemons .shstartdatanode 1.5.配置Yarn 1.5.1.修改文件mapred-site.xml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 <property> <name>mapreduce.framework.name< /name > <value>yarn< /value > < /property > <property> <property> <name>mapreduce.jobhistory.address< /name > <value>hadoop0:10020< /value > < /property > <property> <name>mapreduce.jobhistory.webapp.address< /name > <value>hadoop0:19888< /value > < /property > <property> <name>mapreduce.task.io. sort .factor< /name > <value>20< /value > <description>Thenumberofstreamstomergeatonce while sorting files.Thisdeterminesthenumberof open file handles.< /description > < /property > <property> <name>mapreduce.reduce.shuffle.parallelcopies< /name > <value>40< /value > <description>Thedefaultnumberofparalleltransfersrunbyreduce duringthecopy(shuffle)phase. < /description > < /property > <property> <name>mapreduce.job.reduce.slowstart.completedmaps< /name > <value>0.80< /value > <description>Fractionofthenumberofmaps in thejobwhichshouldbe completebeforereducesarescheduled for thejob. < /description > < /property > <property> <name>mapreduce.task.io. sort .mb< /name > <value>300< /value > <description>Thetotalamountofbuffermemorytousewhilesorting files, in megabytes.Bydefault,giveseachmergestream1MB, which shouldminimizeseeks.< /description > < /property > <property> <name>mapreduce.map.output.compress< /name > <value> true < /value > <description>Shouldtheoutputsofthemapsbecompressedbeforebeing sentacrossthenetwork.UsesSequenceFilecompression. < /description > < /property > <property> <name>mapreduce.client.submit. file .replication< /name > <value>5< /value > <description>默认10，Thereplicationlevel for submittedjobfiles.This shouldbearoundthesquarerootofthenumberofnodes. < /description > < /property > 1.5.2.修改文件yarn-site.xml 1 2 3 4 <property> <name>yarn.resourcemanager.ha.enabled< /name > <value> true < /value > < /property > 【打开resourcemanager ha模式】 1 2 3 4 <property> <name>yarn.resourcemanager.cluster- id < /name > <value>yarn-ha-cluster< /value > < /property > 【打开resourcemanager ha的集群名称，这个名称可以在zookeeper中查看】 1 2 3 4 <property> <name>yarn.resourcemanager.ha. rm -ids< /name > <value>rm1,rm2< /value > < /property > 【设置resourcemanager的id,可以与主机同名】 1 2 3 4 <property> <name>yarn.resourcemanager. hostname .rm1< /name > <value>hadoop0< /value > < /property > 【指定rm1对应哪一台主机】 1 2 3 4 <property> <name>yarn.resourcemanager. hostname .rm2< /name > <value>hadoop1< /value > < /property > 【指定rm1对应哪一台主机】 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 <property> <name>yarn.resourcemanager.zk-address< /name > <value>zk1:2181,zk2:2181,zk3:2181< /value > < /property > <property> <name>yarn.nodemanager.aux-services< /name > <value>mapreduce_shuffle< /value > < /property > <!---用什么方式进行数据传递-> <property> <name>yarn.log-aggregation- enable < /name > <value> true < /value > < /property > 【设置日志合并】 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 <property> <name>yarn.application.classpath< /name > <value> /usr/local/hadoop-2 .5.0-cdh5.3.8 /etc/hadoop , /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/common/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/common/lib/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/hdfs/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/hdfs/lib/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/yarn/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /share/hadoop/yarn/lib/ *, /usr/local/hbase-0 .98.6-cdh5.3.8 /lib/ *, /usr/local/hadoop-2 .5.0-cdh5.3.8 /lib_dc/ *, /usr/local/hbase-0 .98.6-cdh5.3.8 /conf/ < /value > < /property > 【设置classpath,没有新增外部jar的话，不需要配这个】 1 2 3 4 5 6 7 8 9 10 <!--设置调度器类型为CapacityScheduler，默认是公平调度器--> <property> <name>yarn.resourcemanager.scheduler.class< /name ><value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler< /value > < /property > <!--设置nodemanager节点内存大小，CPU个数--> <property> <name>yarn.nodemanager.resource.memory-mb< /name > <value>28672< /value > < /property > 【设置nodemanager节点内存大小 28G】 1 2 3 4 <property> <name>yarn.nodemanager.resource.cpu-vcores< /name > <value>14< /value > < /property > 【设置nodemanager节点内存大小 14个core】 1 2 3 4 <property> <name>yarn.timeline-service.enabled< /name > <value> true < /value > < /property > 【打开timeline服务】 1.5.3 修改环境变量（可以不改，使用默认配置） 1.5.3.1修改 yarn-env.sh 1 YARN_LOG_DIR= /data0/hadoop2/logs 【修改yarn的日志目录，默认在$HADOOP_HOME/logs下】 1.5.3.2 修改hadoop-env.sh 1 export JAVA_HOME= /usr/java/jdk1 .7.0_25 【修改jdk】 1 export HADOOP_LOG_DIR= /data0/hadoop2/logs 【修改hadoop的日志目录，默认在$HADOOP_HOME/logs下】 1 export HADOOP_PID_DIR= /data0/hadoop2/pid 【修改hadoop pid目录】 1.5.5修改capacity-scheduler.xml这个文件（可选 ，设置调度器为CapacityScheduler时，可以通过这个配置文件修改作业队列） 1 2 3 4 5 6 7 8 9 10 11 12 <configuration> <!--comparememroyonly--> <property> <name>yarn.scheduler.capacity.resource-calculator< /name > <value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator< /value > < /property > <!-- more chance for appmasterthatwilllaunchMRjobs--> <property> <name>yarn.scheduler.capacity.maximum-am-resource-percent< /name > <value>0.9< /value > < /property > 【appMaster可以使用集群多少资源】 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 <!--打开异步调度--> <property> <name>yarn.scheduler.capacity.schedule-asynchronously. enable < /name > <value> true < /value > < /property > <!--定义队列名称queues,dev--> <property> <name>yarn.scheduler.capacity.root.queues< /name > <value>default,dev< /value > < /property > <property> <name>yarn.scheduler.capacity.root.dev.capacity< /name > <value>50< /value > < /property > 【设置队列dev的能力大小占集群的50%】 1 2 3 4 <property> <name>yarn.scheduler.capacity.root.default.capacity< /name > <value>50< /value > < /property > 【设置队列default的能力大小占集群的50%】 1 < /configuration > 1.5.4 启动yarn 在hadoop0上执行 1 sbin /start-yarn .sh 本文转自 zouqingyun 51CTO博客，原文链接：http://blog.51cto.com/zouqingyun/1754041，如需转载请自行联系原作者
2017-11-22
优秀的个人博客,低调大师

CentOS 7.4搭建Kubernetes 1.8.5集群

环境介绍 角色 操作系统 IP 主机名 Docker版本 master,node CentOS 7.4 192.168.0.210 node210 17.11.0-ce node CentOS 7.4 192.168.0.211 node211 17.11.0-ce node CentOS 7.4 192.168.0.212 node212 17.11.0-ce 1.基础环境配置(所有服务器执行) a.SELinux关闭 sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/sysconfig/selinux setenforce 0 b.Docker安装 curl -sSL https://get.docker.com/ | sh c.配置国内Docker镜像加速器 curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://e2a6d434.m.daocloud.io d.开启Docker开机自动启动 systemctl enable docker.service systemctl restart docker 2.kubernetes证书准备(master执行) a.为将文件复制到Node节点，节省部署时间，我这里做ssh信任免密复制 ssh-genkey -t rsa ssh-copy-id 192.168.0.211 ssh-copy-id 192.168.0.212 b.下载证书生成工具 yum -y install wget wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 chmod +x cfssl_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 chmod +x cfssljson_linux-amd64 mv cfssljson_linux-amd64 /usr/local/bin/cfssljson wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl-certinfo_linux-amd64 mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo c.CA证书制作 #目录准备 mkdir /root/ssl cd /root/ssl #创建CA证书配置 vim ca-config.json { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "usages": [ "signing", "key encipherment", "server auth", "client auth" ], "expiry": "87600h" } } } } #创建CA证书请求文件 vim ca-csr.json { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "JIANGXI", "L": "NANCHANG", "O": "k8s", "OU": "System" } ] } #生成CA证书和私钥cfssl gencert -initca ca-csr.json | cfssljson -bare ca #创建kubernetes证书签名请求 vim kubernetes-csr.json { "CN": "kubernetes", "hosts": [ "127.0.0.1", "192.168.0.210", #修改成自己主机的IP "192.168.0.211", #修改成自己主机的IP "192.168.0.212", #修改成自己主机的IP "10.254.0.1", "kubernetes", "node210", #修改成自己主机的主机名 "node211", #修改成自己主机的主机名 "node212", #修改成自己主机的主机名 "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "JIANGXI", "L": "JIANGXI", "O": "k8s", "OU": "System" } ] } #生成kubernetes证书及私钥cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes #创建admin证书签名请求 vim admin-csr.json { "CN": "admin", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "JIANGXI", "L": "JIANGXI", "O": "system:masters", "OU": "System" } ] } #生成admin证书及私钥cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin #创建 kube-proxy 证书签名请求 vim kube-proxy-csr.json { "CN": "system:kube-proxy", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "JIANGXI", "L": "JIANGXI", "O": "k8s", "OU": "System" } ] } #生成证书及私钥cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy #分发证书 mkdir -p /etc/kubernetes/ssl cp -r *.pem /etc/kubernetes/ssl cd /etc scp -r kubernetes/ 192.168.0.211:/etc/ scp -r kubernetes/ 192.168.0.212:/etc/ 3.etcd集群安装及配置 a.下载etcd，并分发至节点 wget https://github.com/coreos/etcd/releases/download/v3.2.11/etcd-v3.2.11-linux-amd64.tar.gz tar zxf etcd-v3.2.11-linux-amd64.tar.gz mv etcd-v3.2.11-linux-amd64/etcd* /usr/local/bin scp -r /usr/local/bin/etc* 192.168.0.211:/usr/local/bin/ scp -r /usr/local/bin/etc* 192.168.0.212:/usr/local/bin/ b.创建etcd服务启动文件 vim /usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd \ --name ${ETCD_NAME} \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ --peer-cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --peer-key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \ --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \ --initial-advertise-peer-urls ${ETCD_INITIAL_ADVERTISE_PEER_URLS} \ --listen-peer-urls ${ETCD_LISTEN_PEER_URLS} \ --listen-client-urls ${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \ --advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS} \ --initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN} \ --initial-cluster infra1=https://192.168.0.210:2380,infra2=https://192.168.0.211:2380,infra3=https://192.168.0.212:2380 \ --initial-cluster-state new \ --data-dir=${ETCD_DATA_DIR} Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target c.创建必要的目录 mkdir -p /var/lib/etcd/ mkdir /etc/etcd d.编辑etcd的配置文件 vim /etc/etcd/etcd.conf node210的配置文件/etc/etcd/etcd.conf为 # [member] ETCD_NAME=infra1 ETCD_DATA_DIR="/var/lib/etcd" ETCD_LISTEN_PEER_URLS="https://192.168.0.210:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.0.210:2379" #[cluster] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.210:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.210:2379" node211的配置文件/etc/etcd/etcd.conf为 # [member] ETCD_NAME=infra2 ETCD_DATA_DIR="/var/lib/etcd" ETCD_LISTEN_PEER_URLS="https://192.168.0.211:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.0.211:2379" #[cluster] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.211:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.211:2379" node212的配置文件/etc/etcd/etcd.conf为 # [member] ETCD_NAME=infra3 ETCD_DATA_DIR="/var/lib/etcd" ETCD_LISTEN_PEER_URLS="https://192.168.0.212:2380" ETCD_LISTEN_CLIENT_URLS="https://192.168.0.212:2379" #[cluster] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.212:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.212:2379" #在所有节点执行，启动etcd systemctl daemon-reload systemctl enable etcd systemctl start etcd systemctl status etcd 如果报错，就需要查看/var/log/messages文件进行排错 e.测试集群是否正常 验证ETCD是否成功启动 etcdctl \ --ca-file=/etc/kubernetes/ssl/ca.pem \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ cluster-health 4.配置kubernetes参数 a.下载kubernetes编译好的二进制文件并进行分发 wget https://dl.k8s.io/v1.8.5/kubernetes-server-linux-amd64.tar.gz tar zxf kubernetes-server-linux-amd64.tar.gz cp -rf kubernetes/server/bin/{kube-apiserver,kube-controller-manager,kubectl,kubefed,kubelet,kube-proxy,kube-scheduler} /usr/local/bin/ scp -r kubernetes/server/bin/{kubelet,kube-proxy} 192.168.0.211:/usr/local/bin/ scp -r kubernetes/server/bin/{kubelet,kube-proxy} 192.168.0.212:/usr/local/bin/ #查看kubernetes最新版，可到https://github.com/kubernetes/kubernetes/releases 然后进入 CHANGELOG-x.x.md就可限制二进制的下载地址 b.创建 TLS Bootstrapping Token cd /etc/kubernetes export BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ') cat > token.csv <<EOF ${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap" EOF c.创建 kubelet bootstrapping kubeconfig 文件 cd /etc/kubernetes export KUBE_APISERVER="https://192.168.0.210:6443" #设置集群参数 kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=bootstrap.kubeconfig #设置客户端认证参数 kubectl config set-credentials kubelet-bootstrap \ --token=${BOOTSTRAP_TOKEN} \ --kubeconfig=bootstrap.kubeconfig #设置上下文参数 kubectl config set-context default \ --cluster=kubernetes \ --user=kubelet-bootstrap \ --kubeconfig=bootstrap.kubeconfig #设置默认上下文kubectl config use-context default --kubeconfig=bootstrap.kubeconfig d.创建 kube-proxy kubeconfig 文件export KUBE_APISERVER="https://192.168.0.210:6443" #设置集群参数 kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=kube-proxy.kubeconfig #设置客户端认证参数 kubectl config set-credentials kube-proxy \ --client-certificate=/etc/kubernetes/ssl/kube-proxy.pem \ --client-key=/etc/kubernetes/ssl/kube-proxy-key.pem \ --embed-certs=true \ --kubeconfig=kube-proxy.kubeconfig #设置上下文参数 kubectl config set-context default \ --cluster=kubernetes \ --user=kube-proxy \ --kubeconfig=kube-proxy.kubeconfig #设置默认上下文kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig e.创建 kubectl kubeconfig 文件export KUBE_APISERVER="https://192.168.0.210:6443" #设置集群参数 kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} #设置客户端认证参数 kubectl config set-credentials admin \ --client-certificate=/etc/kubernetes/ssl/admin.pem \ --embed-certs=true \ --client-key=/etc/kubernetes/ssl/admin-key.pem #设置上下文参数 kubectl config set-context kubernetes \ --cluster=kubernetes \ --user=admin #设置默认上下文kubectl config use-context kubernetes f.将2个bootstrap.kubeconfig kube-proxy.kubeconfig文件分发至其余服务器 scp -r *.kubeconfig 192.168.0.211:/etc/kubernetes/ scp -r *.kubeconfig 192.168.0.212:/etc/kubernetes/ 5.MASTER安装及配置 a.apiserver安装配置 #apiserver服务启动文件 vim /usr/lib/systemd/system/kube-apiserver.service [Unit] Description=Kubernetes API Service Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target After=etcd.service [Service] EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/apiserver ExecStart=/usr/local/bin/kube-apiserver \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ $KUBE_ETCD_SERVERS \ $KUBE_API_ADDRESS \ $KUBE_API_PORT \ $KUBELET_PORT \ $KUBE_ALLOW_PRIV \ $KUBE_SERVICE_ADDRESSES \ $KUBE_ADMISSION_CONTROL \ $KUBE_API_ARGS Restart=on-failure Type=notify LimitNOFILE=65536 [Install] WantedBy=multi-user.target #配置kubernetes默认配置 vim /etc/kubernetes/config ### # kubernetes system config # # The following values are used to configure various aspects of all # kubernetes services, including # # kube-apiserver.service # kube-controller-manager.service # kube-scheduler.service # kubelet.service # kube-proxy.service # logging to stderr means we get it in the systemd journal KUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debug KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=true" # How the controller-manager, scheduler, and proxy find the apiserver #KUBE_MASTER="--master=http://sz-pg-oam-docker-test-001.tendcloud.com:8080" KUBE_MASTER="--master=http://192.168.0.210:8080" #配置apiserver参数 vim /etc/kubernetes/apiserver ### ## kubernetes system config ## ## The following values are used to configure the kube-apiserver ## # ## The address on the local server to listen to. #KUBE_API_ADDRESS="--insecure-bind-address=sz-pg-oam-docker-test-001.tendcloud.com" KUBE_API_ADDRESS="--advertise-address=192.168.0.210 --bind-address=192.168.0.210 --insecure-bind-address=192.168.0.210" # ## The port on the local server to listen on. #KUBE_API_PORT="--port=8080" # ## Port minions listen on #KUBELET_PORT="--kubelet-port=10250" # ## Comma separated list of nodes in the etcd cluster KUBE_ETCD_SERVERS="--etcd-servers=https://192.168.0.210:2379,https://192.168.0.211:2379,https://192.168.0.212:2379" # ## Address range to use for services KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # ## default admission control policies KUBE_ADMISSION_CONTROL="--admission-control=ServiceAccount,NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota" # ## Add your own! KUBE_API_ARGS="--authorization-mode=RBAC --runtime-config=rbac.authorization.k8s.io/v1beta1 --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/etc/kubernetes/token.csv --service-node-port-range=30000-32767 --tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem --tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem --client-ca-file=/etc/kubernetes/ssl/ca.pem --service-account-key-file=/etc/kubernetes/ssl/ca-key.pem --etcd-cafile=/etc/kubernetes/ssl/ca.pem --etcd-certfile=/etc/kubernetes/ssl/kuberne #如果出现错误，查看/var/log/messages tes.pem --etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem --enable-swagger-ui=true --apiserver-count=3 --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --audit-log-path=/var/lib/audit.log -- event-ttl=1h" #启动apiserver systemctl daemon-reload systemctl enable kube-apiserver systemctl start kube-apiserver systemctl status kube-apiserver #如果出现错误，查看/var/log/messages b.controller-manager服务配置 #controller-manager服务启动文件 vim /usr/lib/systemd/system/kube-controller-manager.service [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/controller-manager ExecStart=/usr/local/bin/kube-controller-manager \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ $KUBE_MASTER \ $KUBE_CONTROLLER_MANAGER_ARGS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target #配置controller-manager服务配置文件 vim /etc/kubernetes/controller-manager #The following values are used to configure the kubernetes controller-manager #defaults from config and apiserver should be adequate #Add your own! KUBE_CONTROLLER_MANAGER_ARGS="--address=127.0.0.1 --service-cluster-ip-range=10.254.0.0/16 --cluster-name=kubernetes --cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem --cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem --service-account-private-key-file=/etc/kubernetes/ssl/ca-key.pem --root-ca-file=/etc/kubernetes/ssl/ca.pem --leader-elect=true" #启动controller-manager服务 systemctl daemon-reload systemctl enable kube-controller-manager systemctl start kube-controller-manager c.scheduler服务安装及配置 #配置scheduler服务启动文件 vim /usr/lib/systemd/system/kube-scheduler.service [Unit] Description=Kubernetes Scheduler Plugin Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/scheduler ExecStart=/usr/local/bin/kube-scheduler \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ $KUBE_MASTER \ $KUBE_SCHEDULER_ARGS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target #配置scheduler服务配置文件 vim /etc/kubernetes/scheduler #kubernetes scheduler config #default config should be adequate #Add your own! KUBE_SCHEDULER_ARGS="--leader-elect=true --address=127.0.0.1" #启动scheduler服务 systemctl daemon-reload systemctl enable kube-scheduler systemctl start kube-scheduler d.测试master是否正常kubectl get componentstatuses #结果如下说明正常 NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {"health": "true"} etcd-1 Healthy {"health": "true"} etcd-2 Healthy {"health": "true"} 6.node安装(所有节点) a.flannel安装及配置(容器网络我们采用flannel) #yum安装flannelyum install -y flannel #检查node节点证书情况ls /etc/kubernetes/ssl #修改flannel.service配置文件如下 vi /usr/lib/systemd/system/flanneld.service [Unit] Description=Flanneld overlay address etcd agent After=network.target After=network-online.target Wants=network-online.target After=etcd.service Before=docker.service [Service] Type=notify EnvironmentFile=/etc/sysconfig/flanneld EnvironmentFile=-/etc/sysconfig/docker-network ExecStart=/usr/bin/flanneld-start \ -etcd-endpoints=${ETCD_ENDPOINTS} \ -etcd-prefix=${ETCD_PREFIX} \ $FLANNEL_OPTIONS ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker Restart=on-failure [Install] WantedBy=multi-user.target RequiredBy=docker.service #修改flannel配置文件 vi /etc/sysconfig/flanneld # Flanneld configuration options # etcd url location. Point this to the server where etcd runs ETCD_ENDPOINTS="https://192.168.0.210:2379,https://192.168.0.211:2379,https://192.168.0.212:2379" # etcd config key. This is the configuration key that flannel queries # For address range assignment ETCD_PREFIX="/kube-centos/network" # Any additional options that you want to pass FLANNEL_OPTIONS="-etcd-cafile=/etc/kubernetes/ssl/ca.pem -etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem -etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem" #在etcd中创建网络配置 etcdctl --endpoints=https://192.168.0.210:2379,https://192.168.0.211:2379,https://192.168.0.212:2379 \ --ca-file=/etc/kubernetes/ssl/ca.pem \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ mkdir /kube-centos/network etcdctl --endpoints=https://192.168.0.210:2379,https://192.168.0.211:2379,https://192.168.0.212:2379 \ --ca-file=/etc/kubernetes/ssl/ca.pem \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ mk /kube-centos/network/config '{"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}' #flannel服务启动 systemctl daemon-reload systemctl enable flanneld systemctl start flanneld systemctl status flanneld b.配置Docker服务启动文件,集成flannel vim /usr/lib/systemd/system/docker.service 在ExecStart上增加 EnvironmentFile=-/run/flannel/docker EnvironmentFile=-/run/docker_opts.env EnvironmentFile=-/run/flannel/subnet.env 修改如下 ExecStart=/usr/bin/dockerd --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} 效果如下: [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker EnvironmentFile=-/run/flannel/docker EnvironmentFile=-/run/docker_opts.env EnvironmentFile=-/run/flannel/subnet.env ExecStart=/usr/bin/dockerd --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. #TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process # restart the docker process if it exits prematurely Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target #重启启动Docker服务 systemctl daemon-reload systemctl restart docker systemctl status docker 切记要先启动flannel，再启动Docker c.查询etcd是否分配网络 etcdctl --endpoints=${ETCD_ENDPOINTS} \ --ca-file=/etc/kubernetes/ssl/ca.pem \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ ls /kube-centos/network/subnets 结果大致如下 /kube-centos/network/subnets/172.30.1.0-24 /kube-centos/network/subnets/172.30.54.0-24 /kube-centos/network/subnets/172.30.99.0-24 etcdctl --endpoints=${ETCD_ENDPOINTS} \ --ca-file=/etc/kubernetes/ssl/ca.pem \ --cert-file=/etc/kubernetes/ssl/kubernetes.pem \ --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \ get /kube-centos/network/config 结果大致如下 {"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}} d.安装及配置kubelet #创建kubelet服务启动文件 vim /usr/lib/systemd/system/kubelet.service [Unit] Description=Kubernetes Kubelet Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=docker.service Requires=docker.service [Service] WorkingDirectory=/var/lib/kubelet EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/kubelet ExecStart=/usr/local/bin/kubelet \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ $KUBELET_API_SERVER \ $KUBELET_ADDRESS \ $KUBELET_PORT \ $KUBELET_HOSTNAME \ $KUBE_ALLOW_PRIV \ $KUBELET_POD_INFRA_CONTAINER \ $KUBELET_ARGS Restart=on-failure [Install] WantedBy=multi-user.target #kubelet认证配置文件 vim /etc/kubernetes/kubelet.kubeconfig apiVersion: v1 kind: Config clusters: - cluster: server: http://192.168.0.210:8080 name: local contexts: - context: cluster: local name: local current-context: local #kubelet配置文件 vim /etc/kubernetes/kubelet node210下/etc/kubernetes/kubelet内容如下 ### ## kubernetes kubelet (minion) config # ## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=192.168.0.210" # ## The port for the info server to serve on #KUBELET_PORT="--port=10250" # ## You may leave this blank to use the actual hostname KUBELET_HOSTNAME="--hostname-override=192.168.0.210" # ## location of the api-server #KUBELET_API_SERVER="--api-servers=http://192.168.0.210:8080" KUBELET_API_SERVER=" " # ## pod infrastructure container #KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=kubernetes/pause" # ## Add your own! #KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --require-kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false" KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false --fail-swap-on=false" node211里配置文件如下 ### ## kubernetes kubelet (minion) config # ## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=192.168.0.211" # ## The port for the info server to serve on #KUBELET_PORT="--port=10250" # ## You may leave this blank to use the actual hostname KUBELET_HOSTNAME="--hostname-override=192.168.0.211" # ## location of the api-server #KUBELET_API_SERVER="--api-servers=http://192.168.0.210:8080" KUBELET_API_SERVER=" " # ## pod infrastructure container #KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=kubernetes/pause" # ## Add your own! #KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --require-kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false" KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false --fail-swap-on=false" node212里配置文件如下 ### ## kubernetes kubelet (minion) config # ## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=192.168.0.212" # ## The port for the info server to serve on #KUBELET_PORT="--port=10250" # ## You may leave this blank to use the actual hostname KUBELET_HOSTNAME="--hostname-override=192.168.0.212" # ## location of the api-server #KUBELET_API_SERVER="--api-servers=http://192.168.0.210:8080" KUBELET_API_SERVER=" " # ## pod infrastructure container #KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=kubernetes/pause" # ## Add your own! #KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --require-kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false" KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false --fail-swap-on=false" #启动kubelet服务 mkdir -p /var/lib/kubelet systemctl daemon-reload systemctl enable kubelet systemctl start kubelet systemctl status kubelet #这里很容易出错，出错时查看/var/log/messages看日志进行排错 #检查kubelet服务是否正常 kubectl get nodes NAME STATUS ROLES AGE VERSION 192.168.0.210 Ready <none> 14h v1.8.5 192.168.0.211 Ready <none> 14h v1.8.5 192.168.0.212 Ready <none> 14h v1.8.5 c.安装及配置kube-proxy #配置kube-proxy服务启动文件 vim /usr/lib/systemd/system/kube-proxy.service [Unit] Description=Kubernetes Kube-Proxy Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target [Service] EnvironmentFile=-/etc/kubernetes/config EnvironmentFile=-/etc/kubernetes/proxy ExecStart=/usr/local/bin/kube-proxy \ $KUBE_LOGTOSTDERR \ $KUBE_LOG_LEVEL \ $KUBE_MASTER \ $KUBE_PROXY_ARGS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target #kube-proxy配置文件如下： node210: vim /etc/kubernetes/proxy ### # kubernetes proxy config # default config should be adequate # Add your own! KUBE_PROXY_ARGS="--bind-address=192.168.0.210 --hostname-override=192.168.0.210 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.254.0.0/16" node211: vim /etc/kubernetes/proxy ### # kubernetes proxy config # default config should be adequate # Add your own! KUBE_PROXY_ARGS="--bind-address=192.168.0.211 --hostname-override=192.168.0.211 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.254.0.0/16" node212: vim /etc/kubernetes/proxy ### # kubernetes proxy config # default config should be adequate # Add your own! KUBE_PROXY_ARGS="--bind-address=192.168.0.212--hostname-override=192.168.0.212 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.254.0.0/16" #启动kube-proxy服务 systemctl daemon-reload systemctl enable kube-proxy systemctl start kube-proxy systemctl status kube-proxy d.在所有节点默认开启forward为accept vim /usr/lib/systemd/system/forward.service [Unit] Description=iptables forward Documentation=http://iptables.org/ After=network.target docker.service [Service] Type=forking ExecStart=/usr/sbin/iptables -P FORWARD ACCEPT ExecReload=/usr/sbin/iptables -P FORWARD ACCEPT ExecStop=/usr/sbin/iptables -P FORWARD ACCEPT PrivateTmp=true [Install] WantedBy=multi-user.target #启动forward服务 systemctl daemon-reload systemctl enable forward systemctl start forward systemctl status forward 7.测试集群是否工作正常 a.创建一个deploykubectl run nginx --replicas=2 --labels="run=nginx-service" --image=nginx --port=80 b.映射服务到外网可访问kubectl expose deployment nginx --type=NodePort --name=nginx-service c.查看服务状态 kubectl describe svc example-service Name: nginx-service Namespace: default Labels: run=nginx-service Annotations: <none> Selector: run=nginx-service Type: NodePort IP: 10.254.84.99 Port: <unset> 80/TCP NodePort: <unset> 30881/TCP Endpoints: 172.30.1.2:80,172.30.54.2:80 Session Affinity: None Events: <none> d.查看pods启动情况 kubectl get pods NAME READY STATUS RESTARTS AGE nginx-2317272628-nsfrr 1/1 Running 0 1m nginx-2317272628-qbbgg 1/1 Running 0 1m e.在外网通过http://192.168.0.210:30881http://192.168.0.211:30881http://192.168.0.212:30881 都可以访问nginx页面 若无法访问，可通过iptables -nL查看forward链是否开启 本文转自 rong341233 51CTO博客，原文链接:http://blog.51cto.com/fengwan/2049124
2017-11-11

资源下载

更多资源
Mario

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

时间: 2026-10-18 大小: 211.28KB 下载: 56次
Spring

Spring

Spring框架（Spring Framework）是由Rod Johnson于2002年提出的开源Java企业级应用框架，旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念，提供核心容器、应用上下文、数据访问集成等模块，支持整合Hibernate、Struts等第三方框架，其适用范围不仅限于服务器端开发，绝大多数Java应用均可从中受益。

时间: 2026-08-21 大小: 5MB 下载: 2次
Sublime Text

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。

时间: 2026-08-21 大小: 9.84MB 下载: 40次
WebStorm

WebStorm

WebStorm 是jetbrains公司旗下一款JavaScript 开发工具。目前已经被广大中国JS开发者誉为“Web前端开发神器”、“最强大的HTML5编辑器”、“最智能的JavaScript IDE”等。与IntelliJ IDEA同源，继承了IntelliJ IDEA强大的JS部分的功能。

时间: 2025-12-13 大小: 249.54MB 下载: 22次
热门标签
DeepSeek Jdk25 OpenAi HarmonyOS6 Nacos3 SpringBoot4 SpringCloud Docker Kubernetes Service Mesh Redis Gemini3 Rocky

欢迎您来访！

登录后，您将获得更多功能！
热门文章
联系我们

有任何问题或合作意向欢迎联系我们

Email: 99873273@qq.com

QQ: 99873273

用户登录
用户注册