利用Registrator和Consul实现mesos task的服务发现
实现目的: 因为mesos中实际的工作节点是slave,框架marathon启动的任务(容器)都是在随机的slave上执行,所以在每台slave上启动Registrator,用来发现本机上的容器,它会把当前宿主机上的容器自动注册到consul.但是consul找一台salve启动就行,它会把自己选为leader,其他slave上启动Registrator的时候指定此leader就行 环境: 192.168.0.149 Mesos-master、Zookeeper 192.168.0.161 Mesos-master、Zookeeper、Marathon 192.168.0.174 Mesos-master、Zookeeper 192.168.0.239 Mesos-slave、Consul-server、Registrator 192.168.0.236 Mesos-slave、Registrator 部署: 定义IP HOST_IP_1=192.168.0.149 HOST_IP_2=192.168.0.161 HOST_IP_3=192.168.0.174 192.168.0.149 启动Zookeeper [root@master-1~]#dockerrun-d--net="host"-eSERVER_ID=1-eADDITIONAL_ZOOKEEPER_1=server.1=${HOST_IP_1}:2888:3888-eADDITIONAL_ZOOKEEPER_2=server.2=${HOST_IP_2}:2888:3888-eADDITIONAL_ZOOKEEPER_3=server.3=${HOST_IP_3}:2888:3888garland/zookeeper 启动Mesos-master [root@master-1~]#dockerrun--net="host"-p5050:5050-e"MESOS_HOSTNAME=${HOST_IP_1}"-e"MESOS_IP=${HOST_IP_1}"-e"MESOS_ZK=zk://${HOST_IP_1}:2181,${HOST_IP_2}:2181,${HOST_IP_3}:2181/mesos"-e"MESOS_PORT=5050"-e"MESOS_LOG_DIR=/var/log/mesos"-e"MESOS_QUORUM=2"-e"MESOS_REGISTRY=in_memory"-e"MESOS_WORK_DIR=/var/lib/mesos"-dgarland/mesosphere-docker-mesos-master 192.168.0.161 启动Zookeeper [root@master-2~]#dockerrun-d--net="host"-eSERVER_ID=2-eADDITIONAL_ZOOKEEPER_1=server.1=${HOST_IP_1}:2888:3888-eADDITIONAL_ZOOKEEPER_2=server.2=${HOST_IP_2}:2888:3888-eADDITIONAL_ZOOKEEPER_3=server.3=${HOST_IP_3}:2888:3888garland/zookeeper 启动Mesos-master [root@master-2~]#dockerrun--net="host"-p5050:5050-e"MESOS_HOSTNAME=${HOST_IP_2}"-e"MESOS_IP=${HOST_IP_2}"-e"MESOS_ZK=zk://${HOST_IP_1}:2181,${HOST_IP_2}:2181,${HOST_IP_3}:2181/mesos"-e"MESOS_PORT=5050"-e"MESOS_LOG_DIR=/var/log/mesos"-e"MESOS_QUORUM=2"-e"MESOS_REGISTRY=in_memory"-e"MESOS_WORK_DIR=/var/lib/mesos"-dgarland/mesosphere-docker-mesos-master 启动Marathon [root@master-2~]#dockerrun-d-p8080:8080mesosphere/marathonv0.14.0--masterzk://${HOST_IP_1}:2181,${HOST_IP_2}:2181,${HOST_IP_3}:2181/mesos--zkzk://${HOST_IP_1}:2181,${HOST_IP_2}:2181,${HOST_IP_3}:2181/marathon 192.168.0.174 启动Zookeeper [root@master-3~]#dockerrun-d--net="host"-eSERVER_ID=3-eADDITIONAL_ZOOKEEPER_1=server.1=${HOST_IP_1}:2888:3888-eADDITIONAL_ZOOKEEPER_2=server.2=${HOST_IP_2}:2888:3888-eADDITIONAL_ZOOKEEPER_3=server.3=${HOST_IP_3}:2888:3888garland/zookeeper 启动Mesos-master [root@master-3~]#dockerrun--net="host"-p5050:5050-e"MESOS_HOSTNAME=${HOST_IP_3}"-e"MESOS_IP=${HOST_IP_3}"-e"MESOS_ZK=zk://${HOST_IP_1}:2181,${HOST_IP_2}:2181,${HOST_IP_3}:2181/mesos"-e"MESOS_PORT=5050"-e"MESOS_LOG_DIR=/var/log/mesos"-e"MESOS_QUORUM=2"-e"MESOS_REGISTRY=in_memory"-e"MESOS_WORK_DIR=/var/lib/mesos"-dgarland/mesosphere-docker-mesos-master 192.168.0.239 安装Mesos-slave 安装mesosphere源 [root@slave-1~]#rpm-ivh/mesosphere-el-repo-7-1.noarch.rpm 安装mesos [root@slave-1~]#yum-yinstallmesos 配置mesos [root@slave-1~]#echozk://192.168.0.149:2181,192.168.0.161:2181,192.168.0.174:2181/mesos>/etc/mesos/zk [root@slave-1~]#echo192.168.0.239|sudotee/etc/mesos-slave/hostname [root@slave-1~]#echo'docker,mesos'>/etc/mesos-slave/containerizers 启动mesos-slave [root@slave-1~]#systemctlstartmesos-slave&&systemctlenablemesos-slave [root@slave-1~]#systemctldisablemesos-master 启动Consul-server [root@slave-1~]#dockerrun-d--name=consul--net=hostgliderlabs/consul-server-bootstrap-bind=192.168.0.239 NOTE:一个节点启动就行,它会自己把自己选为leader 测试consul-server [root@slave-1~]#curl192.168.0.239:8500/v1/catalog/services {"consul":[]} NOTE:也可以浏览器访问:http://192.168.0.239:8500/ui/ 启动Registrator [root@consul-1~]#dockerrun-d\ --name=registrator\ --net=host\ --volume=/var/run/docker.sock:/tmp/docker.sock\ gliderlabs/registrator:latest\ consul://192.168.0.239:8500 测试Registrator是否跟consul结合把服务自动注册到里面,启动一个redis容器测试 [root@consul-1~]#dockerrun-d-P--name=redisredis [root@consul-1~]#curl192.168.0.239:8500/v1/catalog/services {"consul":[],"redis":[]} consul现在有个服务叫做redis,我们能看到更多关于这个服务的,比如端口、IP [root@consul-1~]#curl$(boot2dockerip):8500/v1/catalog/service/redis[{"Node":"boot2docker","Address":"10.0.2.15","ServiceID":"boot2docker:redis:6379","ServiceName":"redis","ServiceTags":null,"ServiceAddress":"","ServicePort":32768}] 所有slave的容器注册信息都可以在consul中看到 也可以用DNS API测试 [root@slave-1~]#dig@127.0.0.1-p8600redis.service.consul ;<<>>DiG9.9.4-RedHat-9.9.4-18.el7<<>>@127.0.0.1-p8600redis.service.consul ;(1serverfound) ;;globaloptions:+cmd ;;Gotanswer: ;;->>HEADER<<-opcode:QUERY,status:NOERROR,id:937 ;;flags:qraard;QUERY:1,ANSWER:1,AUTHORITY:0,ADDITIONAL:0 ;;WARNING:recursionrequestedbutnotavailable ;;QUESTIONSECTION: ;redis.service.consul. IN A ;;ANSWERSECTION: redis.service.consul. 0 IN A 192.168.0.239 ;;Querytime:1msec ;;SERVER:127.0.0.1#8600(127.0.0.1) ;;WHEN:WedMar3012:43:34CST2016 ;;MSGSIZErcvd:74 参考文章: ·https://github.com/sekka1/mesosphere-docker ·http://gliderlabs.com/registrator/latest/user/quickstart/
