容器编排技术
一、前言
- 了解Jenkins的离线安装步骤
- 掌握Gitlab的使用和管理
- 了解CICD的配置步骤和方法
二、基础环境部署
| 机器 | IP | 作用 |
|---|---|---|
| master | 192.168.100.10 | K8s 的 master 节点、Harbor 节点、CI/CD 节点 |
| node | 192.168.100.20 | K8s 的 node 节点 |
在这两台机器上提前部署好 CentOS 7 + Docker + Kubernetes12 + Harbor3 环境，准备好 XianDian-PaaS-v2.4.iso 包45
1. 安装 Jenkins 环境
1.1 查看 K8s 集群状态和节点信息
$ kubectl get csWarning: v1 ComponentStatus is deprecated in v1.19+NAME STATUS MESSAGE ERRORscheduler Healthy okcontroller-manager Healthy oketcd-0 Healthy$ kubectl get nodesNAME STATUS ROLES AGE VERSIONmaster Ready control-plane 31d v1.27.16node Ready <none> 31d v1.27.161.2 安装 Jenkins
$ docker run -d --name jenkins -p 8080:8080 -u root \-v /home/jenkins_home:/var/jenkins_home \-v /var/run/docker.sock:/var/run/docker.sock \-v $(which docker):/usr/bin/docker \-v /usr/bin/kubectl:/usr/local/bin/kubectl \-v /root/.kube:/root/.kube \jenkins/jenkins:latest-jdk2122f894f46083c2b8836ae4e4c67e0bf4e8cd2e09adcc29417d6b8b473dea1407 然后在电脑浏览器输入 http://192.168.100.10:8080 来访问 Jenkins
然后进入容器，输入密码
$ docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword2169d1cbaa1546f396c0571477d2ce6a 下面的 2169d1cbaa1546f396c0571477d2ce6a 就是我们的密码，输入密码并点击 “继续”
然后点击 “安装推荐的插件”
插件安装完成后自动转入管理员用户创建界面
输入用户信息，点击“保存并完成”
配置完URL后使用新建的用户登录 Jenkins
2. 部署 GitLab
2.1 启动 GitLab
GitLab 是利用 Ruby on Rails 一个开源的版本管理系统，实现一个自托管的 Git 项目仓库，可通过 Web 界面进行访问公开的或者私人项目。
与 Github 类似，GitLab 能够浏览源代码，管理缺陷和注释，可以管理团队对仓库的访问，它非常易于浏览提交过的版本并提供一个文件历史库，团队成员可以利用内置的简单 聊天程序(Wall) 进行交流。Gitlab 还提供一个代码片段收集功能可以轻松实现代码复用，便于日后有需要的时候进行查找
本项目Gitlab与Harbor共用一台服务器
$ docker run -d -h gitlab -p 1022:22 -p 81:80 -p 443:443 \--volume /srv/gitlab/config:/etc/gitlab \--volume /srv/gitlab/gitlab/logs:/var/log/gitlab \--volume /srv/gitlab/gitlab/data:/var/opt/gitlab \--restart always --name mygitlab gitlab/gitlab-ce:lateste87a1e5298f2c0430ad5f2269472e9f06cc2eddf71477fb28a6e8089cf46f8d4 GitLab 启动较慢，可以通过 docker logs 查看启动状态
$ docker logs mygitlab 启动完成后，在浏览器上访问 http://192.168.100.10:81 来进入刚刚搭建的本地 GitLab
然后设置 root 用户信息
然后使用 root 用户登录
2.2 创建项目
点击 “Create a project”(创建项目)，创建项目 “ChinaskillProject”，Visibility Level(可见等级) 设置为 “Public”
点击 “Create Project”(创建项目)，然后自动进入项目
根据提示，配置 SSH 公钥到 GitLab 上
$ ssh-keygen -t rsa$ cat ~/.ssh/id_rsa.pub 然后复制出来的内容，到 GitLab 上
GitLab 上点击 “Add SSH Key”(添加 SSH 密钥)，在 Key 栏里面粘贴你刚才复制的内容，Title 默认即可，Expires at(有效期截至到) 随便选，然后点击 “Add Key”(添加密钥)
然后将 之前准备好的 XianDian-PaaS-v2.4.iso4 上传并挂载到虚拟机， git push 源代码到 GitLab 的 ChinaskillProject 项目
$ sudo yum install -y git$ mount /root/XianDian-PaaS-v2.4.iso /mnt/$ cp -rf /mnt/ChinaskillProject/ /opt$ ls -al /opt/ChinaskillProject/总用量 36dr-xr-xr-x 13 root root 4096 12月 25 11:18 .drwxr-xr-x. 5 root root 60 12月 25 11:18 ..dr-xr-xr-x 3 root root 50 12月 25 11:18 account-servicedr-xr-xr-x 3 root root 50 12月 25 11:18 auth-servicedr-xr-xr-x 3 root root 50 12月 25 11:18 config-r-xr-xr-x 1 root root 823 12月 25 11:18 docker-compose.dev.yml-r-xr-xr-x 1 root root 4020 12月 25 11:18 docker-compose.ymldr-xr-xr-x 3 root root 50 12月 25 11:18 gatewaydr-xr-xr-x 8 root root 185 12月 25 11:18 .git-r-xr-xr-x 1 root root 68 12月 25 11:18 .gitignore-r-xr-xr-x 1 root root 1107 12月 25 11:18 LICENCEdr-xr-xr-x 3 root root 51 12月 25 11:18 mongodbdr-xr-xr-x 3 root root 50 12月 25 11:18 monitoringdr-xr-xr-x 3 root root 50 12月 25 11:18 notification-service-r-xr-xr-x 1 root root 723 12月 25 11:18 pom.xml-r-xr-xr-x 1 root root 1075 12月 25 11:18 README.mddr-xr-xr-x 3 root root 50 12月 25 11:18 registrydr-xr-xr-x 3 root root 50 12月 25 11:18 statistics-service-r-xr-xr-x 1 root root 4428 12月 25 11:18 .travis.ymldr-xr-xr-x 4 root root 35 12月 25 11:18 yaml
$ cd /opt/ChinaskillProject/# 移除“指定远程目录”$ git remote remove origin# 指定“我们自己的”远程目录$ git remote add origin http://192.168.100.10:81/root/chinaskillproject.git
$ git add .$ git commit -m "Initial commit"$ git push -u origin master# 这里输入 rootUsername for 'http://192.168.100.10:81': root# 这里设置你刚才配置 GitLab 的时候配置的密码Password for 'http://root@192.168.100.10:81':Counting objects: 3192, done.Delta compression using up to 8 threads.Compressing objects: 100% (1428/1428), done.Writing objects: 100% (3192/3192), 1.40 MiB | 0 bytes/s, done.Total 3192 (delta 1233), reused 3010 (delta 1207)remote: Resolving deltas: 100% (1233/1233), done.To http://192.168.100.10:81/root/chinaskillproject.git * [new branch] master -> master分支 master 设置为跟踪来自 origin 的远程分支 master。 刷新网页，ChinaskillProject 项目中已经有我们刚刚 git push 上去的文件了
3. 配置 Jenkins 连接 GitLab
3.1 设置 Outbound requests(出站请求)
登录 GitLab，打开 Dashboard，然后点击 🔧(settings)
然后点击 “Settings” → “Network” → “Outbound requests”(出战请求) 旁边的 Expand(展开)
然后勾选 “Allow requests to the local network from web hooks and services”
在下面的输入栏里面输入 192.168.100.0/24
然后点击 “Save change”(保存设置)
3.2 创建 Github API Token
点击 GitLab 的用户头像，然后点击 “Settings” → “Access Token” 添加一个 Token
在 Scopes(范围) 上选上 “api” 和 “read_user”，其他的和上面一样，随便填一个就行，然后点击 “Create personal access token”(创建个人访问令牌)
然后复制 “Your New Personal Access Token”(你的新个人访问令牌) 栏里面的内容，后面配置 Jenkins 的时候需要用到
bWxAexndVn6N-XsmFHWz
3.3 设置 Jenkins
登录 Jenkins Dashboard，然后点击 “Manage Jenkins”(管理 Jenkins/系统管理) → “Plugins”(插件) → “Installed plugins”(已安装插件) 查看有没有以下插件
- “GitLab”
- “Generic Webhook Trigger”
- “GitLab API”
- “GitLab Authentication”
- “GitLab Branch Source”
- “Gitlab Merge Request Builder”
- “GitLab Logo”
如果没有的话就到 “Available plugins”(可用插件) 去安装一下
然后回到 Jenkins Dashboard，点击 “Manage Jenkins”(管理 Jenkins/系统管理) → “System”(系统配置)
然后配置 GitLab 信息，取消勾选 “Enable authentication for ‘/project’ end-point” 然后在 “Credentials” 下面点击 “添加”
然后在 类型 里面选择 “GitLab API token”，“API token” 填入刚刚复制的 token
然后点击 “Test Connection”(测试连接)
4. 配置 Jenkins 连接 maven
4.1 安装 maven
由于我们这里的 Jenkins 是采用 Docker in Docker 的方式启动的，所以需要在 Jenkins 容器内安装 maven
# 下载 maven-3.6.3$ curl -o /home/jenkins_home/apache-maven-3.6.3-bin.tar.gz https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
# 进入 Docker 容器$ docker exec -it jenkins /bin/bash $ tar -zxvf /var/jenkins_home/apache-maven-3.6.3-bin.tar.gz -C . $ mv apache-maven-3.6.3/ /usr/local/maven $ apt update -y && apt upgrade -y $ apt install -y vim $ vim ~/.bashrc # 文末加入 export M2_HOME=/usr/local/maven export PATH=$PATH:$M2_HOME/bin
$ source ~/.bashrc $ exit# 退出容器重新进入$ docker exec -it jenkins /bin/bash $ mvn -v Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Maven home: /usr/local/maven Java version: 21.0.5, vendor: Eclipse Adoptium, runtime: /opt/java/openjdk Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "3.10.0-1160.71.1.el7.x86_64", arch: "amd64", family: "unix"4.2 连接 maven
登录 Jenkins，点击 “Manage Jenkins”(管理 Jenkins/系统管理) → “Tools”(工具/全局系统配置)
往下翻找到 Maven，点击“新增Maven”，此处不要开启自动安装，所以不要勾选“自动安装”
然后把填入 Maven 路径填入 MAVEN_HOME ，应用保存即可
5. 部署 Harbor 仓库
5.1 安装 Harbor 仓库
Harbor 依赖 Docker 和 Docker Compose。因此，首先需要在系统中安装 Docker Compose
$ DOCKER_CONFIG=/usr/local/lib/docker/cli-plugins$ sudo mkdir -p $DOCKER_CONFIG/cli-plugins$ sudo curl -SL https://github.com/docker/compose/releases/download/v2.29.1/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose$ sudo chmod +x $DOCKER_CONFIG/cli-plugins$ docker compose version 然后下载 Offline 版的 Harbor
$ sudo curl -O https://github.com/goharbor/harbor/releases/download/v2.12.1/harbor-offline-installer-v2.12.1.tgz
# 国内用户可以使用以下方式加快下载$ sudo curl -O https://moka.anitsuri.top//image/13/Registry+Harbor/harbor-offline-installer-v2.11.1.tgz
$ sudo tar -zxf harbor-offline-installer-v2.12.1.tgz$ cd harbor 然后修改配置文件，把 harbor.yml.tmpl 复制一个变成 harbor.yml
$ sudo cp harbor.yml.tmpl harbor.yml$ sudo vi harbor.yml 修改/填写以下内容
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.hostname: 192.168.100.10
# http related confighttp: # port for http, default is 80. If https enabled, this port will redirect to https port port: 80
# https related config# https: # https port for harbor, default is 443 # port: 443 # The path of cert and key files for nginx # certificate: /your/certificate/path # private_key: /your/private/key/path # enable strong ssl ciphers (default: false) # strong_ssl_ciphers: false
# # Harbor will set ipv4 enabled only by default if this block is not configured# # Otherwise, please uncomment this block to configure your own ip_family stacks# ip_family:# # ipv6Enabled set to true if ipv6 is enabled in docker network, currently it affected the nginx related component# ipv6:# enabled: false# # ipv4Enabled set to true by default, currently it affected the nginx related component# ipv4:# enabled: true
# # Uncomment following will enable tls communication between all harbor components# internal_tls:# # set enabled to true means internal tls is enabled# enabled: true# # put your cert and key files on dir# dir: /etc/harbor/tls/internal
# Uncomment external_url if you want to enable external proxy# And when it enabled the hostname will no longer used# external_url: http://192.168.92.128:8433
# The initial password of Harbor admin# It only works in first time to install harbor# Remember Change the admin password from UI after launching Harbor.harbor_admin_password: Harbor12345 在运行安装脚本前，需要先运行 prepare 脚本，来确保一切依赖和配置都已经就绪
$ ./prepareprepare base dir is set to /root/harborUnable to find image 'goharbor/prepare:v2.12.1' locallyv2.12.1: Pulling from goharbor/prepare0e5c20819ebf: Pull complete1fb69a2a4ee3: Pull completec2aef12efe8a: Pull complete79fdf0882382: Pull completeeee2f1d222f1: Pull complete125a453a0ce4: Pull complete40b43804af0b: Pull complete5979e246bb31: Pull complete8d2c370a2c28: Pull complete53eaa20ee0df: Pull completeDigest: sha256:d0fcec61328185ddb5bfded6bf8b312428aa63b9bffe2a8d846452253234a898Status: Downloaded newer image for goharbor/prepare:v2.12.1WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to httpsGenerated configuration file: /config/portal/nginx.confGenerated configuration file: /config/log/logrotate.confGenerated configuration file: /config/log/rsyslog_docker.confGenerated configuration file: /config/nginx/nginx.confGenerated configuration file: /config/core/envGenerated configuration file: /config/core/app.confGenerated configuration file: /config/registry/config.ymlGenerated configuration file: /config/registryctl/envGenerated configuration file: /config/registryctl/config.ymlGenerated configuration file: /config/db/envGenerated configuration file: /config/jobservice/envGenerated configuration file: /config/jobservice/config.ymlGenerated and saved secret to file: /data/secret/keys/secretkeySuccessfully called func: create_root_certGenerated configuration file: /compose_location/docker-compose.ymlClean up the input dir
$ ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 26.1.4
[Step 1]: checking docker-compose is installed ...
Note: Docker Compose version v2.27.1
[Step 2]: loading Harbor images ...581136816168: Loading layer [==================================================>] 17.21MB/17.21MBcbf4f3cb0c26: Loading layer [==================================================>] 3.584kB/3.584kB19ba001b8b03: Loading layer [==================================================>] 2.56kB/2.56kBb41af00dcd76: Loading layer [==================================================>] 70.97MB/70.97MB5b5f57643342: Loading layer [==================================================>] 5.632kB/5.632kB5175984e7e7a: Loading layer [==================================================>] 126.5kB/126.5kBc8faf279c61c: Loading layer [==================================================>] 201.7kB/201.7kB3a0151c1eba9: Loading layer [==================================================>] 72.09MB/72.09MBdbf86e000945: Loading layer [==================================================>] 2.56kB/2.56kBLoaded image: goharbor/harbor-core:v2.12.17d2be923c4b3: Loading layer [==================================================>] 146.8MB/146.8MB1b7e2d8bb9c9: Loading layer [==================================================>] 3.584kB/3.584kBc433c2b34c5a: Loading layer [==================================================>] 3.072kB/3.072kB902158d6f915: Loading layer [==================================================>] 2.56kB/2.56kBe795d71404bd: Loading layer [==================================================>] 3.072kB/3.072kBcec684dd57a5: Loading layer [==================================================>] 3.584kB/3.584kB47d0e4a2c5c7: Loading layer [==================================================>] 20.48kB/20.48kBLoaded image: goharbor/harbor-log:v2.12.1594f33e86793: Loading layer [==================================================>] 22.04MB/22.04MBe0a4bafe73dc: Loading layer [==================================================>] 191.7MB/191.7MB26b746af8db7: Loading layer [==================================================>] 26.16MB/26.16MB31dcafd9cb03: Loading layer [==================================================>] 18.54MB/18.54MBabf08c52686b: Loading layer [==================================================>] 5.12kB/5.12kB0007d565ec69: Loading layer [==================================================>] 6.144kB/6.144kB6e863404971d: Loading layer [==================================================>] 3.072kB/3.072kB94c8eba384cc: Loading layer [==================================================>] 2.048kB/2.048kB1a5955f6795e: Loading layer [==================================================>] 2.56kB/2.56kB8ce6cc317185: Loading layer [==================================================>] 7.68kB/7.68kBLoaded image: goharbor/harbor-db:v2.12.171775c4c621c: Loading layer [==================================================>] 17.21MB/17.21MB4003ab14e211: Loading layer [==================================================>] 3.584kB/3.584kBbeabc71b1223: Loading layer [==================================================>] 2.56kB/2.56kB579b2f017459: Loading layer [==================================================>] 59.9MB/59.9MB895f989740a0: Loading layer [==================================================>] 60.69MB/60.69MBLoaded image: goharbor/harbor-jobservice:v2.12.19590068fe044: Loading layer [==================================================>] 14.26MB/14.26MB0cb250c20318: Loading layer [==================================================>] 4.096kB/4.096kB9dac56331be1: Loading layer [==================================================>] 18.14MB/18.14MB5dc6f164dc78: Loading layer [==================================================>] 3.072kB/3.072kB141d394a5d47: Loading layer [==================================================>] 37.13MB/37.13MB2587acb702ad: Loading layer [==================================================>] 56.05MB/56.05MBLoaded image: goharbor/harbor-registryctl:v2.12.1dc1a7f40e575: Loading layer [==================================================>] 17.21MB/17.21MB3f56a5b9b5d4: Loading layer [==================================================>] 37.15MB/37.15MBd67a91463720: Loading layer [==================================================>] 4.608kB/4.608kBcedc4b1b129a: Loading layer [==================================================>] 37.94MB/37.94MBLoaded image: goharbor/harbor-exporter:v2.12.137959c9a5eac: Loading layer [==================================================>] 137.2MB/137.2MBLoaded image: goharbor/nginx-photon:v2.12.1d07b5c3ce0a9: Loading layer [==================================================>] 137.2MB/137.2MBf60722775a24: Loading layer [==================================================>] 6.731MB/6.731MB4c9a05314a30: Loading layer [==================================================>] 252.9kB/252.9kB61258323c496: Loading layer [==================================================>] 1.497MB/1.497MBLoaded image: goharbor/harbor-portal:v2.12.19a1a8734fa3c: Loading layer [==================================================>] 14.75MB/14.75MB856558c960ff: Loading layer [==================================================>] 4.096kB/4.096kB9bd63d8a60d9: Loading layer [==================================================>] 3.072kB/3.072kBa89a7d8b1691: Loading layer [==================================================>] 133.9MB/133.9MB008e693070f3: Loading layer [==================================================>] 15.55MB/15.55MBcef35bccdc97: Loading layer [==================================================>] 150.3MB/150.3MBLoaded image: goharbor/trivy-adapter-photon:v2.12.1d17edfb68476: Loading layer [==================================================>] 22.06MB/22.06MBa0f8af0fdb08: Loading layer [==================================================>] 126.6MB/126.6MB27e661303e23: Loading layer [==================================================>] 3.072kB/3.072kB3f0390c3788e: Loading layer [==================================================>] 59.9kB/59.9kBf56e18460c59: Loading layer [==================================================>] 61.95kB/61.95kBLoaded image: goharbor/redis-photon:v2.12.18b7f3238d8aa: Loading layer [==================================================>] 14.26MB/14.26MB295711278366: Loading layer [==================================================>] 4.096kB/4.096kBddadeb03375d: Loading layer [==================================================>] 3.072kB/3.072kB99bbf20549c8: Loading layer [==================================================>] 18.14MB/18.14MB9237fc270c32: Loading layer [==================================================>] 18.93MB/18.93MBLoaded image: goharbor/registry-photon:v2.12.1Loaded image: goharbor/prepare:v2.12.1
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...prepare base dir is set to /root/harborWARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to httpsClearing the configuration file: /config/portal/nginx.confClearing the configuration file: /config/log/logrotate.confClearing the configuration file: /config/log/rsyslog_docker.confClearing the configuration file: /config/nginx/nginx.confClearing the configuration file: /config/core/envClearing the configuration file: /config/core/app.confClearing the configuration file: /config/registry/passwdClearing the configuration file: /config/registry/config.ymlClearing the configuration file: /config/registryctl/envClearing the configuration file: /config/registryctl/config.ymlClearing the configuration file: /config/db/envClearing the configuration file: /config/jobservice/envClearing the configuration file: /config/jobservice/config.ymlGenerated configuration file: /config/portal/nginx.confGenerated configuration file: /config/log/logrotate.confGenerated configuration file: /config/log/rsyslog_docker.confGenerated configuration file: /config/nginx/nginx.confGenerated configuration file: /config/core/envGenerated configuration file: /config/core/app.confGenerated configuration file: /config/registry/config.ymlGenerated configuration file: /config/registryctl/envGenerated configuration file: /config/registryctl/config.ymlGenerated configuration file: /config/db/envGenerated configuration file: /config/jobservice/envGenerated configuration file: /config/jobservice/config.ymlloaded secret from file: /data/secret/keys/secretkeyGenerated configuration file: /compose_location/docker-compose.ymlClean up the input dir
Note: stopping existing Harbor instance ...
[Step 5]: starting Harbor ...[+] Running 10/10 ✔ Network harbor_harbor Created 0.3s ✔ Container harbor-log Started 2.0s ✔ Container registryctl Started 2.9s ✔ Container redis Started 2.9s ✔ Container harbor-db Started 3.0s ✔ Container harbor-portal Started 3.0s ✔ Container registry Started 3.0s ✔ Container harbor-core Started 3.2s ✔ Container nginx Started 3.8s ✔ Container harbor-jobservice Started 3.7s✔ ----Harbor has been installed and started successfully.----5.2 配置 Harbor 到 Docker Daemon 文件
在两台机器上，编辑 /etc/docker/daemon.json 文件，把以下内容复制/添加进去
{ "insecure-registries": [ "192.168.100.10" ]} 然后重新加载 daemon 并重新启动 Docker
$ sudo systemctl daemon-reload$ sudo systemctl restart docker 然后耐心等待 Jenkins 和 GitLab 的容器恢复
5.3 关闭 containerd 的 HTTPS 请求
编辑 两台主机 的 /etc/containerd/config.toml 文件，修改或添加以下内容
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.100.10"] endpoint = ["http://192.168.100.10"] 然后重启 containerd 服务
$ sudo systemctl restart containerd6. 配置 CI/CD
6.1 新建任务
登录 Jenkins Dashboard，点击 “新建Item”(新建任务)，选择构建一个流水线，然后 “确定”
然后在 “构建触发器” 一栏，勾选 “Build when a change is pushed to GitLab. GitLab webhook URL: http://192.168.100.10:8080/project/ChinaskillProject↗”
记录下 GitLab Webhook URL的地址(http://192.168.100.10:8080/project/ChinaskillProject)，后期配置 Webhook 需要使用
然后配置下方 “流水线”，然后点击下面的 “流水线语法”：“示例步骤” 选择 “git：Git”，将 ChinaskillProject 项目的 GitLab 地址填入“仓库 URL”
然后点一下 “凭据” 旁边的 “添加” 按钮，选择 “Jenkins”，类型选择 “Username with password”，用户名和密码为 GitLab 仓库的用户名和密码
添加凭据之后选择凭据，最后是这样的
然后点击 “生成流水线脚本”
然后记录生成的值
git credentialsId: '757b55dd-2337-4703-808d-bbf147610cd4', url: 'http://192.168.100.10:81/root/chinaskillproject.git' 将其写入下面的流水线脚本里面
这里是完整的流水线脚本
node { stage('git clone') { // Check CODE git credentialsId: '757b55dd-2337-4703-808d-bbf147610cd4', url: 'http://192.168.100.10:81/root/chinaskillproject.git' } stage('maven build') { sh '''/usr/local/maven/bin/mvn package -DskipTests -f /var/jenkins_home/workspace/ChinaskillProject''' } stage('image build') { sh ''' echo $BUILD_ID docker build -t 192.168.100.10/chinaskillproject/gateway:$BUILD_ID -f /var/jenkins_home/workspace/ChinaskillProject/gateway/Dockerfile /var/jenkins_home/workspace/ChinaskillProject/gateway docker build -t 192.168.100.10/chinaskillproject/config:$BUILD_ID -f /var/jenkins_home/workspace/ChinaskillProject/config/Dockerfile /var/jenkins_home/workspace/ChinaskillProject/config ''' } stage('test') { sh ''' docker run -itd --name gateway 192.168.100.10/chinaskillproject/gateway:$BUILD_ID docker ps -a | grep chinaskillproject | grep Up if [ $? -eq 0 ]; then echo "Success!" docker rm -f gateway else docker rm -f gateway exit 1 fi ''' } stage('upload registry') { sh ''' docker login 192.168.100.10 -u=admin -p=Harbor12345 docker push 192.168.100.10/chinaskillproject/gateway:$BUILD_ID docker push 192.168.100.10/chinaskillproject/config:$BUILD_ID ''' } stage('deploy Rancher') { // 执行部署脚本 sh 'sed -i "s/sqshq\\/piggymetrics-gateway/192.168.100.10\\/chinaskillproject\\/gateway:$BUILD_ID/g" /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/gateway-deployment.yaml' sh 'sed -i "s/sqshq\\/piggymetrics-config/192.168.100.10\\/chinaskillproject\\/config:$BUILD_ID/g" /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/config-deployment.yaml' sh 'kubectl create ns ChinaskillProject' sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/gateway-deployment.yaml --kubeconfig=/root/.kube/config' sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/config-deployment.yaml --kubeconfig=/root/.kube/config' sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config' sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config' }} 在 “脚本” 一栏里写入脚本之后即可点击 “Save” 或者是 “应用”
6.2 开启 Jenkins 匿名访问
登录 Jenkins Dashboard ，点击 “Manage Jenkins”(管理 Jenkins/系统管理) → “Security”(安全/全局安全配置)，然后配置授权策略允许匿名用户访问
6.3 配置 Webhook
登录 GitLab，进入 “ChinaskillProject” 项目
点击 “Settings” → “Webhooks”，将前面记录的 GitLab Webhook URL 地址填入 URL 处，并禁用 SSL 认证
点击 “Add webhook” 添加 Webhook
然后点击 “Test”(测试) → “Push events”(推送活动) 进行测试
结果返回 HTTP 200 则表明 Webhook 配置成功
6.4 创建仓库项目
登录 Harbor，新建项目 chinaskillproject，访问级别设置为 “公开”
进入项目查看镜像列表，可以看到此时为空，无任何镜像
三、正式构建
触发 CI/CD
注意
在此之前，需要把 XianDian-PaaS-v2.4.iso 里面的 ./ChinaskillProject 项目下所有目录里面的 Dockerfile 的 java:8-jre 全都改成 openjdk:8-jre，然后重新上传 GitLab 仓库，否则会导致构建失败
解决方法之一：
在修改完内容之后，新文件夹是
/root/ChinaskillProjectTerminal window $ cd /root/ChinaskillProject$ git remote remove origin$ git remote add origin http://192.168.100.10:81/root/chinaskillproject.git$ git add .$ git commit -m "Initial commit"$ git push -u origin master -f然后刷新 GitLab，看 commit 时间是最近的说明成功
如果提示错误，就到 GitLab 的 ChinaskillProject 项目里，进入 “Settings”(设置) → ""
1. 触发构建
回到 GitLab 上的 “ChinaskillProject” 项目，点击 “Settings” → “Webhooks” 往下翻找到 “Test”(测试) 然后点击 “Push events”(推送活动) 即可正常构建
或者你也可以上传代码，来自动触发构建
$ docker cp /opt/repository/ jenkins:/root/.m2/$ cd /opt/ChinaskillProject/$ git add .$ git commit -m "Initial commit"$ git push -u origin masterUsername for 'http://192.168.100.10:81': rootPassword for 'http://root@192.168.100.10:81':Counting objects: 3068, done.Delta compression using up to 8 threads.Compressing objects: 100% (1358/1358), done.Writing objects: 100% (3068/3068), 1.39 MiB | 0 bytes/s, done.Total 3068 (delta 1236), reused 3013 (delta 1207)remote: Resolving deltas: 100% (1236/1236), done.To http://192.168.100.10:81/root/chinaskillproject.git + 1b2901a...f44d317 master -> master (forced update)2. Jenkins 查看
登录 Jenkins Dashboard，可以看到 ChinaskillProject 项目已经开始构建
点击项目名称查看流水线阶段视图，点击左边最新的#（我这边是#2）里面的 “Console Output”(控制台输出) 可以看到详细的构建进程
3. Harbor 查看
进入 Harbor 仓库 chinaskillproject 项目查看镜像列表，可以看到已经自动上传了一个 gateway 和 config 镜像
4. Kubernertes 查看
Pod 的启动比较慢，需要等待 3~5 分钟，具体的时间需要看你分配的配置
我们可以在命令行查看 Pod
$ kubectl -n ChinaskillProject get podsNAME READY STATUS RESTARTS AGEconfig-54cf699d8d-6pvx7 1/1 Running 0 2m29sgateway-855db6cd7d-vhsqx 1/1 Running 1 (2m7s ago) 2m30s 查看 Service
$ kubectl -n ChinaskillProject get serviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEconfig NodePort 10.96.109.132 <none> 8888:30015/TCP 24mgateway NodePort 10.106.190.223 <none> 4000:30010/TCP 24m 然后访问 http://192.168.100.10:30010 访问服务
Footnotes
Docker + Kubernetes 可以根据这个脚本一键安装：AliYUM+Docker+K8s_CN.sh ↩
如果不可以，可以试试这个脚本（*需要海外网络环境）：AliYUM+Docker+K8s.sh ↩
如果不会搭建的可以查看【二、5】 ↩
Thanks for reading!