基于Kubernetes/Jenkins/Harbor/Sonarqube搭建微服务CICD系统
**项目说明** 14个JAVA spring cloud 微服务项目 5个Node项目 1个Node项目VUE生成静态页面做前端+nginx pgsql redis rabbitmq emqx 项目思维导图 **构建流程：** 一、项目软件搭建 1，K8S搭建（参考阿良部署文档：kubeadmin安装单主K8S集群） 2，jenkins搭建 https://jenkins.io/doc/book/installing/#fedora 3，Harbor搭建 https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md 4，Sonarqube搭建 sonarqube检测java项目：https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Maven sonarqube检测node项目：https://www.npmjs.com/package/sonarqube-scanner 二、CICD系统中使用的资料 1，jenkins pipeline（JAVA服务的） node{ defmvnHome=tool'Maven' REGISTRY='hb.shubing.vip/k12soft' //environment{ //REGISTRY='hb.shubing.vip/k12soft' //} stage('PullCodefromGitlab.'){ gitbranch:'test_4',credentialsId:'1',url:'http://10.0.0.82:9999/k12soft/xxx.git' } stage('MavenPackageandSonar.'){ withMaven(jdk:'JDK'){ //defJAVA_HOME="/usr/lib/jdk1.8.0_121" sh"'${mvnHome}/bin/mvn'cleanpackage-e-U-Dmaven.test.skip=truesonar:sonar" //sh"'${mvnHome}/bin/mvn'-febase-api/pom.xmlcleanpackagesonar:sonar" } } stage('BuildDockerImage.'){ sh"sudodockerbuild-t$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}." } stage('PushDockerImagetoHarbor.'){ sh"sudodockerpush$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}" } stage('DeployK8SService.'){ sh"sudossh'root@10.0.0.102''cd/root/xtcloud&&shapply-config.sh${BUILD_ID}${JOB_BASE_NAME}'" dingTalkaccessToken:'https://oapi.dingtalk.com/robot/send?access_token=8241e530e0f2e68f74da6b78f893e2f526db4086a88e76de7348ddda101f13xx',imageUrl:'',jenkinsUrl:'',message:"构建&K8S发布成功！",notifyPeople:'' } } 2，jenkins pipeline（Node服务的） node{ REGISTRY='hb.shubing.vip/k12soft' stage('PullCodefromGitlab.'){ gitbranch:'zangyunji',credentialsId:'1',url:'http://10.0.0.82:9999/k12soft/xxx.git' } //stage('NodeInstallModules.'){ //sh"source/etc/profile.d/node.sh&&npmi" //} stage('NodeBuildandSonar.'){ sh"source/etc/profile.d/node.sh&&sonar-scanner" sh"source/etc/profile.d/node.sh&&npmruntsc" } stage('BuildDockerImage.'){ sh"sudodockerbuild-t$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}." } stage('PushDockerImagetoHarbor.'){ sh"sudodockerpush$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}" } stage('DeployK8SService.'){ sh"sudossh'root@10.0.0.102''cd/root/xtcloud&&shapply-config.sh${BUILD_ID}${JOB_BASE_NAME}'" dingTalkaccessToken:'https://oapi.dingtalk.com/robot/send?access_token=8241e530e0f2e68f74da6b78f893e2f526db4086a88e76de7348ddda101f13xx',imageUrl:'',jenkinsUrl:'',message:"构建&K8S发布成功！",notifyPeople:'' } } 3，dockerfile （JAVA服务的） FROMjava:8u111 MAINTAINERgongshubing@weds.com.cn #定义变量 ENVWORK_DIR/opt ENVLOG_DIR/data/logs EXPOSE8086 #从主机copy到容器里面 COPYtarget/bepf_archives.jar$WORK_DIR WORKDIR$WORK_DIR ENTRYPOINT["sh","-c"] CMD["java-server-Dspring.profiles.active=ops-Xmx512m-Xms256m-Xmn256m-XX:+UseG1GC-XX:+DisableExplicitGC-Duser.timezone=GMT+8-jar*.jar>/dev/null" ] 4，dockerfile（NODE服务的） FROMnode:10.15.3 MAINTAINERgongshubing@weds.com.cn #定义变量 ENVWORK_DIR/opt ENVLOG_DIR/data/logs ENVEGG_SERVER_ENVops EXPOSE8086 #从主机copy到容器里面 COPY./$WORK_DIR/ RUNsed-i's/\-\-daemon//g'$WORK_DIR/package.json WORKDIR$WORK_DIR ENTRYPOINT["sh","-c"] CMD["npmstart"] 5，K8S 编排yaml文件（所有项目编排文件一样，不同之处只有镜像URL/IP/PORT）：pod中的日志文件写入nfs #deployment --- apiVersion:apps/v1 kind:Deployment metadata: name:xtcloud-appgateway-deployment labels: app:xtcloud-appgateway spec: replicas:1 selector: matchLabels: app:xtcloud-appgateway template: metadata: labels: app:xtcloud-appgateway spec: containers: -name:xtcloud-appgateway image:hb.shubing.vip/k12soft/xtcloud-appgateway:latest ports: -containerPort:8097 volumeMounts: -name:xtcloud-logs-persistent-storage mountPath:/data/logs volumes: -name:xtcloud-logs-persistent-storage nfs: path:/nfs server:10.0.60.67 imagePullSecrets: -name:weds-harbor-secret #service --- kind:Service apiVersion:v1 metadata: name:xtcloud-appgateway-service spec: clusterIP:192.168.1.15 selector: app:xtcloud-appgateway ports: -protocol:TCP port:8097 targetPort:8097 6，jenkins调用编排K8s脚本 #!/bin/bash TAG=$1 JOB_NAME=$2 #sed"s/latest/$TAG/"${JOB_NAME}.yml|kubectlapply-f- sed"s/latest/$TAG/"${JOB_NAME}.yml|kubectlreplace--force-f- 二、效果图
