1 项目概述
1.1 核心目标
本项目将完成“公有云 K8s 集群搭建 -> 应用容器化 -> CI/CD 流水线构建 -> 自动化部署与验证”的全流程落地，最终实现：
- 开发者提交代码后，自动触发编译、镜像构建、镜像扫描、K8s 部署
- 流水线可观测性（日志、告警、部署记录）
- 自动化：减少人工干预，提高部署效率，降低人为错误。
- 可追溯：每个版本的代码、镜像、部署配置均可追溯。
- 可扩展：支持微服务架构、多集群部署、灰度发布。
1.2 技术栈选型
| 组件类别 | 选型说明 |
|---|---|
| 公有云平台 | 阿里云 ACK |
| Kubernetes | 云厂商托管版（1.24+，减少集群运维成本） |
| CI/CD 工具 | GitLab Jenkins ArgoCD |
| 镜像仓库 | 阿里云 ACR |
| 基础工具 | Docker（容器构建）、Git（代码管理）、kubectl（K8s 命令行） |
2 环境准备
2.1 阿里云容器服务ACK创建
2.1.1 开通阿里云ACK服务
- 登录阿里云控制台 ->搜索 容器服务 Kubernetes 版（ACK）->开通服务
- 进入 集群管理 -> 创建集群，选择 智能托管模式（推荐，无需管理 Master 节点）
2.1.2 集群配置（关键参数）
| 配置项 | 推荐配置 |
|---|---|
| 集群名称 | prod-ack-cluster |
| Kubernetes 版本 | 1.34.3-aliyun.1 |
| 网络模式 | VPC 网络（自动创建）华南1（深圳） |
| 节点配置 | 自动配置 默认是2节点 4核 8 G |
| 容器运行时 | Containerd（默认） |
| 附加组件 | 日志服务、报警配置 |
2.1.3 集群访问配置
1.集群创建完成后，进入 集群详情 -> 连接信息
2.复制内容，并配置本地kubectl
mkdir-p ~/.kubevim~/.kube/config 粘贴内容# 验证连接root@ack:~# kubectl get nodeNAME STATUS ROLES AGE VERSION cn-shenzhen.10.249.9.151 Ready<none>111m v1.34.3-aliyun.1 cn-shenzhen.10.249.9.152 Ready<none>111m v1.34.3-aliyun.1注意：ACK集群默认是不公开公网的，这时候如果想本地连接有以下两种方案
| 对比维度 | 方案一：同 VPC 内 ECS 作为管理节点 | 方案二：ACK 开通公网，本地直接连接 |
|---|---|---|
| 安全性 | 高（内网隔离，仅 ECS 为入口，可控性强） | 低（公网暴露 APIServer，易遭攻击） |
| 合规性 | 符合企业内网化要求 | 多数企业禁止 |
| 成本 | 低（轻量 ECS 约 30-50 元 / 月，按需启停） | 无额外 ECS 成本 |
| 稳定性 | 高（内网低延迟，不受公网波动影响） | 一般（依赖公网，可能卡顿 / 中断） |
| 长期使用 | 适合（可部署管理工具，效率高） | 不推荐（安全风险累积，易出问题） |
| 适用场景 | 企业生产 / 测试集群、长期管理、敏感数据 | 个人测试集群、临时调试（用完即关） |
我是采用在同vpc内创建一台ECS作为管理节点
需要安装kubectl
# 更新apt包索引apt-getupdate# 安装k8s apt仓库需要的包apt-getinstall-y apt-transport-https ca-certificatescurlgpg# 下载用于k8s软件包仓库的公共签名密钥curl-fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key|sudogpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg# 添加 K8s apt 仓库echo'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /'|sudotee/etc/apt/sources.list.d/kubernetes.list# 更新 apt 包索引，安装 kubelet、kubeadm 和 kubectl，并锁定版本sudoapt-getupdatesudoapt-getinstall-y kubectl# 命令补全# 安装bash-completionaptupdate&&aptinstall-y bash-completion# 将kubectl补全写入bashrc配置文件echo"source <(kubectl completion bash)">>~/.bashrc# 生效配置source~/.bashrc2.2 容器镜像仓库 ACR 创建
2.2.1 开通阿里云 ACR 服务
1.控制台搜索 容器镜像服务（ACR）->开通服务
2.创建 个人版 私有仓库：
- 仓库名称：自定义demo-app-repo
- 仓库类型：私有
- 地域：与 K8s 集群地域一致
2.2.2 配置仓库访问凭证
1.进入 ACR 仓库 -> 访问凭证 -> 启用 固定密码
- 记录用户名（阿里云账号 ID）、密码（固定密码），后续 CI/CD 流水线需用此凭证推送镜像。
2.3 部署核心服务
测试环境为了更好的可观测效果，使用nginx作为实际业务镜像
root@ack:~# docker pull nginx:latestlatest: Pulling from library/nginx a2abf6c4d29d: Pull complete a9edb18cadd1: Pull complete 589b7251471a: Pull complete 186b1aaa4aa6: Pull complete b4df32aa5a72: Pull complete a0bcbecc962e: Pull complete Digest: sha256:0d17b565c37bcbd895e9d92315a05c1c3c9a29f762b011a10c54a66cd53c9b31 Status: Downloaded newer imagefornginx:latest docker.io/library/nginx:latest root@ack:~# docker tag nginx:latest crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/ack-test-repo/demo-app-repo:v1root@ack:~# docker push crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/ack-test-repo/demo-app-repo:v1The push refers to repository[crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/ack-test-repo/demo-app-repo]d874fd2bc83b: Pushed 32ce5f6a5106: Pushed f1db227348d0: Pushed b8d6e692a25e: Pushed e379e8aedd4d: Pushed 2edcec3590a4: Pushed v1: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size:15702.3.1 部署服务
1.ConfigMap 配置（Nginx主页文件）
root@ack:~# mkdir yaml/test-serviceroot@ack:~/yaml# cd test-service/root@ack:~/yaml/test-service# vim test-service-welcome-cm.yamlapiVersion: v1 kind: ConfigMap metadata: name: welcome-nginx-cm namespace: test-service data: index.html:|<!DOCTYPE html><html><head><title>Welcome</title></head><body><h1>v1</h1></body></html>2.创建PVC（持久化存储）
# 查看ACK集群默认自带的scroot@ack:~/yaml/test-service# kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE alicloud-disk-efficiency diskplugin.csi.alibabacloud.com Delete Immediatetrue45m alicloud-disk-essd diskplugin.csi.alibabacloud.com Delete Immediatetrue45m alicloud-disk-ssd diskplugin.csi.alibabacloud.com Delete Immediatetrue45m alicloud-disk-topology-alltype diskplugin.csi.alibabacloud.com Delete WaitForFirstConsumertrue45m root@ack:~/yaml/test-service# vim test-service-pvc.yamlapiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-service-pvc namespace: test-service spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi storageClassName: alicloud-disk-essd3.部署主服务
# 创建命名空间root@ack:~/yaml/test-service# kubectl create ns test-servicenamespace/test-service created# 需要先创建凭证否则无权限拉取私有镜像kubectl create secret docker-registry acr-pull-secret\--namespace=test-service\--docker-server=crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com\--docker-username=用户名\--docker-password='密码'# 编写deployment的yaml文件root@ack:~/yaml/test-service# vim test-service-deploy.yamlapiVersion: apps/v1 kind: Deployment metadata: name: test-service namespace: test-service spec: replicas:3selector: matchLabels: app: test-service# 滚动更新配置strategy: type: RollingUpdate rollingUpdate: maxSurge:1maxUnavailable:0template: metadata: labels: app: test-service spec: imagePullSecrets: - name: acr-pull-secret# ACR密钥affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution:# 硬亲和性nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname# 按节点名匹配（指定只调度到node2）operator: In values: - cn-shenzhen.10.105.190.34 containers: - name: test-service image: crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/ack-test-repo/demo-app-repo:v1 ports: - containerPort:80# 挂载ConfigMapvolumeMounts: - name: welcome-page mountPath: /usr/share/nginx/html/index.html subPath: index.html - name: test-data mountPath: /data resources: limits: cpu: 500m memory: 512Mi requests: cpu: 200m memory: 256Mi# 健康检查livenessProbe: httpGet: path: / port:80initialDelaySeconds:5periodSeconds:10readinessProbe: httpGet: path: / port:80initialDelaySeconds:3periodSeconds:5volumes: - name: welcome-page configMap: name: welcome-nginx-cm items: - key: index.html path: index.html - name: test-data persistentVolumeClaim: claimName: test-service-pvc4.创建svc暴露服务端口
root@ack:~/yaml/test-service# vim test-service-svc.yamlapiVersion: v1 kind: Service metadata: name: test-service-loadbalancer namespace: test-service# 阿里云SLB注解annotations: service.beta.kubernetes.io/alicloud-loadbalancer-type:"public"# 公网SLBspec: type: LoadBalancer selector: app: test-service ports: - port:80targetPort:80name: http5.应用全部查看状态
root@ack:~/yaml/test-service# kubectl apply -f .root@ack:~/yaml/test-service# kubectl get pod -n test-serviceNAME READY STATUS RESTARTS AGE test-service-556655549d-65b441/1 Running03m8s test-service-556655549d-9w5nj1/1 Running02m57s test-service-556655549d-trzqh1/1 Running03m2s root@ack:~/yaml/test-service# kubectl get svc -n test-serviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)AGE test-service-loadbalancer LoadBalancer192.168.168.358.135.37.3580:30427/TCP 12m root@ack:~/yaml/test-service# kubectl get pvc -n test-serviceNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE test-service-pvc Bound d-wz9ijd1pkjrw4oibqqnp 20Gi RWO alicloud-disk-essd<unset>9m3 CI/CD链路搭建
3.1 部署Gitlab
3.1.1 获取Gitlab镜像并上传ACR
root@ack:~# docker pull gitlab/gitlab-ce:14.3.6-ce.014.3.6-ce.0: Pulling from gitlab/gitlab-ce 7b1a6ab2e44d: Pull complete db71597928d9: Pull complete 01d82f2a1736: Pull complete 1463060c448a: Pull complete a8e2c0106e8f: Pull complete ddc5b7231755: Pull complete 868d8c1123e4: Pull complete 99882f6311c9: Pull complete Digest: sha256:7a34eed19d7af89e96e52231305171b1f690e99d01fffd4be39c515fd4af1ece Status: Downloaded newer imageforgitlab/gitlab-ce:14.3.6-ce.0 docker.io/gitlab/gitlab-ce:14.3.6-ce.0 root@ack:~# docker login --username=aliyun7958465660 crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.comPassword: WARNING!Your credentials are stored unencryptedin'/root/.docker/config.json'.Configure a credential helper to remove this warning. See https://docs.docker.com/go/credential-store/ root@ack:~# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZE gitlab/gitlab-ce14.3.6-ce.0 7e6628afc6ac4years ago2.29GB root@ack:~# docker tag 7e6628afc6ac crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images:gitlab-ce14.3.6-ce.0root@ack:~# docker push crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images:gitlab-ce14.3.6-ce.0The push refers to repository[crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images]93f7839a2c2b: Layer already exists 41410a33de93: Layer already exists e80e799d5aca: Layer already exists 5614a9a4a82c: Layer already exists 8b1cc33ab454: Layer already exists b8a8b46a8c5d: Layer already exists 181353e4ecef: Layer already exists 9f54eef41275: Layer already exists gitlab-ce14.3.6-ce.0: digest: sha256:7a34eed19d7af89e96e52231305171b1f690e99d01fffd4be39c515fd4af1ece size:1992# 因为宿主机是containerd的运行时没用docker.sock可以挂载# 创建一个sidecar，用来jenkins构建镜像root@ack:~/yaml/jenkins# docker pull docker:dinddind: Pulling from library/docker 59bf1c3509f3: Pull complete 1ea03e1895df: Pull complete 1ff98835b055: Pull complete a3f2dd7b7d65: Pull complete d182b62d4a35: Pull complete d7a57db2abd7: Pull complete 73490af52bd3: Pull complete 7d28806efd0e: Pull complete b97fd78563ff: Pull complete ffaf1aff7f0b: Pull complete b8eed8b92328: Pull complete Digest: sha256:1f50d3a86f7035805843779f803e81e8f4ce96b62ed68fc70cdcf4922f43470b Status: Downloaded newer imagefordocker:dind docker.io/library/docker:dind root@ack:~/yaml/jenkins# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZE crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images jenkins2.528.2 aa2bbbd632f32months ago 490MB jenkins/jenkins2.528.2 aa2bbbd632f32months ago 490MBdockerdind 175122d895c34years ago 233MB crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images gitlab-ce14.3.6-ce.0 7e6628afc6ac4years ago2.29GB gitlab/gitlab-ce14.3.6-ce.0 7e6628afc6ac4years ago2.29GB root@ack:~/yaml/jenkins# docker tag 175122d895c3 crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images:docker-bindroot@ack:~/yaml/jenkins# docker push crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images:docker-bindThe push refers to repository[crpi-36asyostst5a0uzg.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images]cfee98210a2e: Pushed c5b13bae086f: Pushed a0d564da2c93: Pushed ea14261e8ed8: Pushed f796896d9c53: Pushed 106a41e6c9a4: Pushed 3550aa9fea69: Pushed 1320e38b009f: Pushed f7674e51a4a5: Pushed 661ef83a114f: Pushed 8d3ac3489996: Pushed docker-bind: digest: sha256:4e04836731b7100e8bd5e0b35756f53d0b6211ddb3cc7ec326ae3640adcfa004 size:26143.1.2 gitlab-StatefulSet.yaml 配置
root@ack:~/yaml/gitlab# vim gitlab-StatefulSet.yaml# 1. 创建独立命名空间apiVersion: v1 kind: Namespace metadata: name: gitlab labels: name: gitlab ---# 2. Headless Service（必须）：适配StatefulSet的Pod身份标识apiVersion: v1 kind: Service metadata: name: gitlab-headless namespace: gitlab spec: clusterIP: None# 无头服务标识selector: app: gitlab ports: - port:80targetPort:80name: http - port:22targetPort:22name:ssh---# 3. Service：通过阿里云SLB暴露服务apiVersion: v1 kind: Service metadata: name: gitlab-loadbalancer namespace: gitlab# 可选：阿里云SLB注解annotations: service.beta.kubernetes.io/alicloud-loadbalancer-type:"public"# 公网SLB# service.beta.kubernetes.io/alicloud-loadbalancer-instance-charge-type: "PayAsYouGo" # 按量付费spec: type: LoadBalancer selector: app: gitlab ports: - port:80targetPort:80name: http - port:22targetPort:22name:ssh---# 4. StatefulSetapiVersion: apps/v1 kind: StatefulSet metadata: name: gitlab namespace: gitlab spec: serviceName: gitlab-headless# 关联Headless Servicereplicas:1# 测试环境单副本selector: matchLabels: app: gitlab template: metadata: labels: app: gitlab spec: containers: - name: gitlab image: crpi-36asyostst5a0uzg-vpc.cn-shenzhen.personal.cr.aliyuncs.com/repo-test-k8s/public-images:gitlab-ce14.3.6-ce.0# 建议开启资源限制：GitLab资源消耗较高#resources:# limits:
