首页 文章 精选 留言 我的

精选列表

搜索[集群],共10007篇文章
优秀的个人博客,低调大师

Kubernates集群入门(1)

一、K8s安装准备 1.至少两台主机,一台作为master,一台作为node。两台主机需要关闭防火墙。 #centos6 service stop firewalld && service disable firewalld #centos7 systemctl stop iptables && systemctl disable iptables; 2.两台机器需要各自编辑/etc/hosts文件,互相添加hostname,然后相互ping通,以下为例 echo "192.168.18.128 centos-master 192.168.18.130 centos-minion " >> /etc/hosts 二、K8s的安装 1.两台主机都需要安装docker,kubernetes,如有docker版本冲突需要卸载重新安装docker. yum -y install docker kubernetes 2.master节点需要安装etcd数据库服务,etcd作为kubernetes的数据库 yum -y install etcd 3.每个节点,master及minion节点都需要修改kubernetes配置文件 vim /etc/kubernetes/config # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://centos-master:8080" #master节点关于指向etcd的ip可能需要改成127.0.0.1:2379,改成主机名的话kube-controller-manager可能会启动失败,不知原因 KUBE_ETCD_SERVERS="--etcd_servers=http://centos-master:2379" 示例master 下config # kubernetes system config # # The following values are used to configure various aspects of all # kubernetes services, including # # kube-apiserver.service # kube-controller-manager.service # kube-scheduler.service # kubelet.service # kube-proxy.service # logging to stderr means we get it in the systemd journal KUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debug KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=false" # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://kube01:8080" KUBE_ETCD_SERVERS="--etcd-servers=http://kube01:2379" 4.master节点上,配置api服务给node vim /etc/kubernetes/apiserver # The address on the local server to listen to. #这个地址好像只能用0.0.0.0 KUBE_API_ADDRESS="--address=0.0.0.0" KUBE_API_PORT="--port=8080" # Comma separated list of nodes in the etcd cluster #KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:2379" ##ServiceAccount这个参数删掉,会影响docker拉去镜像 # default admission control policies KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" 示例:master节点apiserver: # kubernetes system config # # The following values are used to configure the kube-apiserver # # The address on the local server to listen to. KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" # The port on the local server to listen on. KUBE_API_PORT="--port=8080" # Port minions listen on # KUBELET_PORT="--kubelet-port=10250" # Comma separated list of nodes in the etcd cluster KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379" # Address range to use for services KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # default admission control policies KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota" # Add your own! KUBE_API_ARGS="" 5.master节点上编写启动相关kubernetes服务的脚本 vim k8s-server.sh #!/bin/bash OPT=$1 case $1 in -s) for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done ;; -k) for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler ; do systemctl stop $SERVICES done ;; -stat) for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl status $SERVICES done ;; *) echo "useage:./k8s-server.sh <-s|-k|-stat>---- '-s' is start Servers\n--- '-k' is stop Servers\n'-stat' is watch the status " ;; esac 6.node节点修改/etc/kubernetes/kubelet,配置与master的连接 ### # kubernetes kubelet (minion) config KUBELET_ADDRESS="--address=0.0.0.0" KUBELET_PORT="--port=10250" KUBELET_HOSTNAME="--hostname_override=centos-minion" KUBELET_API_SERVER="--api_servers=http://centos-master:8080“ # Add your own! KUBELET_ARGS="" minion节点 config示例 ### # kubernetes system config # # The following values are used to configure various aspects of all # kubernetes services, including # # kube-apiserver.service # kube-controller-manager.service # kube-scheduler.service # kubelet.service # kube-proxy.service # logging to stderr means we get it in the systemd journal KUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debug KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=false" # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://kube01:8080" minion节点kubelet示例 ### # kubernetes kubelet (minion) config # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=0.0.0.0" # The port for the info server to serve on #KUBELET_PORT="--port=10250" # You may leave this blank to use the actual hostname KUBELET_HOSTNAME="--hostname-override=kube02" # location of the api-server KUBELET_API_SERVER="--api-servers=http://kube01:8080" # pod infrastructure container KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" # Add your own! KUBELET_ARGS="" 7.node节点编写启动和查看服务脚本 #!/bin/bash OPT=$1 case $1 in -s) for SERVICES in kube-proxy kubelet docker; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done ;; -k) for SERVICES in kube-proxy kubelet docker; do systemctl stop $SERVICES done ;; -stat) for SERVICES in kube-proxy kubelet docker; do systemctl status $SERVICES done ;; *) echo "useage:./k8s.sh <-s|-k|-stat>---- '-s' is start Servers\n--- '-k' is stop Servers\n'-stat' is watch the status " ;; esac 8.node节点查看是否成功注册到master节点,如果没关闭防火墙会报错 tail -f /var/log/messages |grep kube 9.master节点查看刚才注册的节点,节点status为ready为正常 kubectl get nodes 10.kubectl是master端的交互工具,可以通过子命令查看节点等信息 kubectl get nodes #获取节点列表 kubectl cluster-info #查看节点信息 下一节演示一个简单的kubernetes实例,master节点通过yaml文件,让node节点自动pull镜像并运行。** 如果启动docker报错,如下 当前docker版本1.13.1 执行启动命令: systemctl start docker ,报下面错误: Error starting daemon: SELinux is not supported with the overlay2 graph driver on this kernel. Either boot into a newer kernel or disable selinux in docker (--selinux-enabled=false) 重新编辑docker配置文件: vi /etc/sysconfig/docker # /etc/sysconfig/docker # Modify these options if you want to change the way the docker daemon runs OPTIONS='--selinux-enabled=false --log-driver=journald --signature-verification=false' if [ -z "${DOCKER_CERT_PATH}" ]; then DOCKER_CERT_PATH=/etc/docker fi :wq systemctl restart docker

优秀的个人博客,低调大师

storm集群的监控

所谓兵马未动,粮草先行,准备将storm用在某个项目中做实时数据分析。无论任何系统,一定要有监控系统并存,当故障发生的时候你能第一个知道,而不是让别人告诉你,那处理故障就很被动了。 因此我写了这么个项目,取名叫storm-monitor,放在了github上https://github.com/killme2008/storm-monitor 主要功能如下:1.监控supervisor数目是否正确,当supervisor挂掉的时候会发送警告。2.监控nimbus是否正常运行,monitor会尝试连接nimbus,如果连接失败就认为nimbus挂掉。3.监控topology是否正常运行,包括它是否正常部署,是否有运行中的任务。 当故障发生的时候通过alarm方法警告用户,开放出去的只是简单地打日志。因为每个公司的告警接口不一样,所以你需要自己扩展,修改alarm.clj即可。我们这儿就支持旺旺告警和手机短信告警。 基本的原理很简单,对supervisor和topology的监控是通过zookeeper来间接地监控,通过定期查看path是否存在。对nimbus的监控是每次起一个短连接连上去,连不上去即认为挂掉。 整个项目也是用clojure写。你的机器需要安装lein和exec插件,然后将你的storm.yaml拷贝到conf目录下,编辑monitor.yaml设定监控参数如检查间隔等,最后启动start.sh脚本即可。默认日志输出在logs/monitor.log。 本文来源于"阿里中间件团队播客",原文发表时间"2011-12-02"

优秀的个人博客,低调大师

HBase集群安装方法

注意HBase与Hadoop兼容性问题 安装jdk aptitude install openjdk-7-jdk 创建hbase用户 for i in AP-HB1 AP-HB2 AP-HB3; do echo =====$i=====; ssh $i "groupadd hbase; useradd -m hbase -g hbase -s /bin/bash -d /home/hbase; passwd hbase"; done 创建HDFS存储目录 for i in AP-HB1 AP-HB2 AP-HB3; do ssh $i "mkdir -p /data/HadoopData/; chown -R hbase:hbase /data/HadoopData"; done 安装HDFS 解压hadoop tar zxf ~/hadoop-1.2.1-bin.tar.gz 配置hadoop-env.sh(指定JAVA_HOME) export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/ 配置core-site.xml <property> <name>hadoop.tmp.dir</name> <value>/data/HadoopData/tmp</value> </property> <property> <name>fs.default.name</name> <value>hdfs://AP-HB1:9000</value> </property> 配置conf/hdfs-site.xml <property> <name>dfs.name.dir</name> <value>/data/HadoopData/dfs/name/</value> </property> <property> <name>dfs.data.dir</name> <value>/data/HadoopData/dfs/data/</value> </property> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.permissions</name> <value>false</value> </property> 配置masters(指定SecondaryNameNode机器) 配置slaves(指定DataNode机器) 安装HBase 解压HBase tar zxf ~/hbase-0.98.12-hadoop1-bin.tar.gz 配置hbase-env.sh(指定JAVA_HOME) export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/ 配置hdfs-site.xml <property> <name>hbase.rootdir</name> <value>hdfs://AP-HB1:9000/hbase</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.master</name> <value>AP-HB1:60000</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/data/HadoopData/zookeeper</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>AP-HB1,AP-HB2,AP-HB3</value> </property> <property> <name>hbase.regionserver.export.thrift</name> <value>true</value> </property> <property> <name>hbase.regionserver.thrift.port</name> <value>9091</value> </property> 配置regionservers AP-HB1 AP-HB2 AP-HB3 复制程序到所有机器 cd /opt; chown hbase.hbase hadoop-1.2.1 hbase-0.98.12-hadoop1 -R for i in AP-HB2 AP-HB3;do rsync -av /opt/hadoop-1.2.1 /opt/hbase-0.98.12-hadoop1 $i:/opt/; done 设置ssh免密登录 for i in AP-HB1 AP-HB2 AP-HB3;do ssh $i "mkdir /home/hbase/.ssh; ssh-keygen -t rsa -P '' -f /home/hbase/.ssh/id_rsa; chown hbase.hbase /home/hbase/.ssh -R"; done for i in AP-HB2 AP-HB3;do ssh $i "cat /home/hbase/.ssh/authorized_keys";done >> /home/hbase/.ssh/authorized_keys for i in AP-HB2 AP-HB3;do scp /home/hbase/.ssh/authorized_keys $i:/home/hbase/.ssh/;done 开启HDFS服务 cd /opt/hadoop-1.2.1; bin/start-dfs.sh 开启HBase服务 cd /opt/hbase-0.98.12-hadoop1; bin/start-hbase.sh Troubleshooting 2015-05-19 16:48:34,815 ERROR org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Error getting localhost name. Using 'localhost'...hadoop要使用hostname,需要修改hostname与配置文件一致,修改hostname hostname $hostname cat $hostname > /etc/hostname 2015-05-19 17:40:00,771 FATAL [AP-HB1:16020.activeMasterManager] master.HMaster: Unhandled exception. Starting shutdown. java.io.IOException: Failed on local exception: java.io.IOException: Broken pipe; Host Details : local host is: "AP-HB1/10.162.50.249"; destination host is: "AP-HB1":9000;注意HBase与Hadoop版本兼容性问题

资源下载

更多资源
优质分享App

优质分享App

近一个月的开发和优化,本站点的第一个app全新上线。该app采用极致压缩,本体才4.36MB。系统里面做了大量数据访问、缓存优化。方便用户在手机上查看文章。后续会推出HarmonyOS的适配版本。

腾讯云软件源

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题,腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构,目前腾讯云软件源站支持公网访问和内网访问。

Rocky Linux

Rocky Linux

Rocky Linux(中文名:洛基)是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版,作为CentOS稳定版停止维护后与RHEL(Red Hat Enterprise Linux)完全兼容的开源替代方案,由社区拥有并管理,支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性,采用模块化包装和SELinux安全架构,默认包含GNOME桌面环境及XFS文件系统,支持十年生命周期更新。

Sublime Text

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能,例如代码缩略图,Python的插件,代码段等。还可自定义键绑定,菜单和工具栏。Sublime Text 的主要功能包括:拼写检查,书签,完整的 Python API , Goto 功能,即时项目切换,多选择,多窗口等等。Sublime Text 是一个跨平台的编辑器,同时支持Windows、Linux、Mac OS X等操作系统。

用户登录
用户注册