1. 项目背景与核心价值
在当今云原生技术栈中,Kubernetes已成为容器编排的事实标准,而Docker Compose和Helm作为不同层次的部署工具,各自在开发和生产环境中扮演着重要角色。RHEL 8作为企业级Linux发行版,其稳定性和安全性使其成为运行关键业务负载的首选平台。
这个技术方案的核心价值在于打通了从开发到生产的部署链路:
- 开发阶段使用Docker Compose快速构建和测试微服务
- 生产环境通过Helm将服务打包为标准的Kubernetes应用
- 整个过程在RHEL 8上实现无缝衔接
我曾在一个电商平台迁移项目中采用这套方案,将原本需要2天的部署流程缩短到2小时内完成,且实现了开发环境与生产环境的高度一致。
2. 环境准备与工具链配置
2.1 RHEL 8基础环境调优
在开始之前,我们需要对RHEL 8进行一些必要的配置优化:
bash复制# 禁用防火墙(仅限实验环境,生产环境需配置规则)
sudo systemctl stop firewalld
sudo systemctl disable firewalld
# 设置SELinux为permissive模式
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# 安装基础工具链
sudo dnf install -y vim git curl wget tar gzip
注意:生产环境中不建议完全禁用防火墙和SELinux,而应该配置适当的规则和策略
2.2 Docker与Docker Compose安装
RHEL 8默认使用Podman而非Docker,我们需要手动添加Docker CE仓库:
bash复制# 添加Docker CE仓库
sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
# 安装Docker CE
sudo dnf install -y docker-ce docker-ce-cli containerd.io
# 启动并设置开机自启
sudo systemctl enable --now docker
# 验证安装
sudo docker run hello-world
安装Docker Compose(建议使用v2.x版本):
bash复制# 下载最新稳定版
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
# 设置执行权限
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
# 验证安装
docker compose version
2.3 Kubernetes与Helm安装
对于开发测试环境,我们可以使用Minikube来部署本地Kubernetes集群:
bash复制# 安装Minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm
sudo rpm -Uvh minikube-latest.x86_64.rpm
# 启动集群(需要提前安装kubectl)
minikube start --driver=docker
# 安装kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# 安装Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
3. 微服务开发与Docker Compose编排
3.1 示例微服务架构设计
我们以一个典型的电商微服务为例,包含以下组件:
- 前端服务(Node.js)
- 商品服务(Spring Boot)
- 订单服务(Python Flask)
- Redis缓存
- PostgreSQL数据库
目录结构建议:
code复制microservices/
├── frontend/
├── product-service/
├── order-service/
└── docker-compose.yml
3.2 Docker Compose文件详解
docker-compose.yml的完整配置:
yaml复制version: '3.8'
services:
frontend:
build: ./frontend
ports:
- "3000:3000"
depends_on:
- product-service
- order-service
environment:
- NODE_ENV=development
- PRODUCT_SERVICE_URL=http://product-service:8080
- ORDER_SERVICE_URL=http://order-service:5000
product-service:
build: ./product-service
ports:
- "8080:8080"
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/products
- SPRING_DATASOURCE_USERNAME=postgres
- SPRING_DATASOURCE_PASSWORD=postgres
depends_on:
- db
- redis
order-service:
build: ./order-service
ports:
- "5000:5000"
environment:
- DATABASE_URL=postgresql://postgres:postgres@db:5432/orders
- REDIS_URL=redis://redis:6379
depends_on:
- db
- redis
redis:
image: redis:alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
db:
image: postgres:13-alpine
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=postgres
- POSTGRES_DB=products
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
volumes:
redis_data:
postgres_data:
3.3 开发工作流实践
- 启动所有服务:
bash复制docker compose up -d --build
- 查看服务日志:
bash复制docker compose logs -f frontend
- 执行测试:
bash复制# 进入容器执行测试
docker compose exec product-service ./mvnw test
- 停止服务:
bash复制docker compose down
经验分享:在开发过程中,可以使用
docker compose watch功能(Compose v2.17+)实现代码热重载,大幅提升开发效率
4. Helm Chart设计与Kubernetes部署
4.1 从Docker Compose到Helm的转换策略
将Docker Compose服务迁移到Helm需要以下转换思路:
-
服务拆分:
- 每个微服务对应一个Helm子Chart
- 共享服务(如Redis、PostgreSQL)作为独立Chart或使用Bitnami等公共Chart
-
配置管理:
- 将docker-compose.yml中的environment转换为Helm values.yaml
- 网络配置转换为Kubernetes Service和Ingress
-
存储管理:
- 将volumes转换为PersistentVolumeClaim
- 考虑使用StorageClass动态配置
4.2 Helm Chart目录结构
标准的Helm Chart结构:
code复制ecommerce/
├── Chart.yaml
├── values.yaml
├── charts/
├── templates/
│ ├── frontend/
│ ├── product-service/
│ ├── order-service/
│ ├── redis/
│ └── postgresql/
└── templates/tests/
4.3 关键模板示例
- Deployment模板(templates/frontend/deployment.yaml):
yaml复制apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Chart.Name }}-frontend
labels:
app: {{ .Chart.Name }}-frontend
spec:
replicas: {{ .Values.frontend.replicaCount }}
selector:
matchLabels:
app: {{ .Chart.Name }}-frontend
template:
metadata:
labels:
app: {{ .Chart.Name }}-frontend
spec:
containers:
- name: frontend
image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}"
imagePullPolicy: {{ .Values.frontend.image.pullPolicy }}
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: {{ .Values.frontend.env.NODE_ENV | quote }}
- name: PRODUCT_SERVICE_URL
value: {{ printf "http://%s-product-service:%d" .Chart.Name .Values.productService.service.port | quote }}
resources:
{{- toYaml .Values.frontend.resources | nindent 12 }}
- Service模板(templates/frontend/service.yaml):
yaml复制apiVersion: v1
kind: Service
metadata:
name: {{ .Chart.Name }}-frontend
spec:
type: {{ .Values.frontend.service.type }}
ports:
- port: {{ .Values.frontend.service.port }}
targetPort: 3000
protocol: TCP
name: http
selector:
app: {{ .Chart.Name }}-frontend
4.4 Values.yaml配置示例
yaml复制# Global values
global:
env: production
# Frontend configuration
frontend:
replicaCount: 2
image:
repository: myrepo/frontend
tag: latest
pullPolicy: IfNotPresent
service:
type: ClusterIP
port: 3000
env:
NODE_ENV: production
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 256Mi
# Product Service configuration
productService:
# ... similar structure ...
5. 自动化部署流水线实现
5.1 CI/CD流程设计
完整的自动化部署流程应包含以下阶段:
- 代码提交触发构建
- 容器镜像构建与推送
- Helm Chart版本更新
- Kubernetes部署验证
- 自动化测试
- 生产环境发布
5.2 GitHub Actions实现示例
.github/workflows/deploy.yml配置:
yaml复制name: Deploy Microservices
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# 登录Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
# 构建并推送前端镜像
- name: Build and push frontend
working-directory: frontend
run: |
docker build -t ${{ secrets.DOCKER_HUB_USERNAME }}/ecommerce-frontend:$GITHUB_SHA .
docker push ${{ secrets.DOCKER_HUB_USERNAME }}/ecommerce-frontend:$GITHUB_SHA
# 构建并推送其他服务镜像...
# 安装Helm
- name: Install Helm
uses: azure/setup-helm@v3
with:
version: 'v3.11.1'
# 配置kubectl
- name: Configure kubectl
uses: azure/setup-kubectl@v3
# 更新Helm Chart
- name: Update Helm Chart
run: |
git clone https://github.com/myorg/helm-charts.git
cd helm-charts/ecommerce
# 更新values.yaml中的镜像tag
yq e ".frontend.image.tag = \"$GITHUB_SHA\"" -i values.yaml
git config user.name "GitHub Actions"
git config user.email "actions@github.com"
git commit -am "Update image tags for $GITHUB_SHA"
git push
# Helm部署到测试环境
- name: Deploy to Staging
run: |
helm upgrade --install ecommerce ./helm-charts/ecommerce \
--namespace staging \
--create-namespace \
--values ./helm-charts/ecommerce/values.yaml \
--set frontend.image.tag=$GITHUB_SHA
5.3 Argo CD集成(可选)
对于更复杂的GitOps工作流,可以集成Argo CD:
- 安装Argo CD:
bash复制kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
- 创建Application CRD:
yaml复制apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ecommerce
namespace: argocd
spec:
destination:
server: https://kubernetes.default.svc
namespace: production
source:
repoURL: https://github.com/myorg/helm-charts.git
path: ecommerce
targetRevision: HEAD
helm:
values: |
frontend:
replicaCount: 3
syncPolicy:
automated:
prune: true
selfHeal: true
6. 生产环境注意事项
6.1 安全加固措施
-
容器安全:
- 使用非root用户运行容器
- 设置readOnlyRootFilesystem: true
- 配置适当的securityContext
-
网络策略:
yaml复制apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: frontend-policy spec: podSelector: matchLabels: app: ecommerce-frontend policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: ecommerce-ingress ports: - protocol: TCP port: 3000 egress: - to: - podSelector: matchLabels: app: ecommerce-product-service ports: - protocol: TCP port: 8080 -
密钥管理:
- 使用Kubernetes Secrets或外部密钥管理系统(如HashiCorp Vault)
- 避免在values.yaml中直接存储敏感信息
6.2 监控与日志方案
- Prometheus监控配置示例:
yaml复制apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: frontend-monitor
spec:
selector:
matchLabels:
app: ecommerce-frontend
endpoints:
- port: http
interval: 30s
path: /metrics
- 集中式日志收集(Fluentd配置示例):
yaml复制<source>
@type tail
path /var/log/containers/*frontend*.log
pos_file /var/log/fluentd-frontend.log.pos
tag kube.frontend
<parse>
@type json
time_format %Y-%m-%dT%H:%M:%S.%NZ
</parse>
</source>
6.3 性能调优技巧
- 资源配额管理:
yaml复制apiVersion: v1
kind: ResourceQuota
metadata:
name: ecommerce-quota
spec:
hard:
requests.cpu: "8"
requests.memory: 16Gi
limits.cpu: "16"
limits.memory: 32Gi
- HPA自动扩缩配置:
yaml复制apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: frontend-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ecommerce-frontend
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
7. 常见问题排查指南
7.1 部署问题排查流程
- 检查Pod状态:
bash复制kubectl get pods -n <namespace>
- 查看Pod日志:
bash复制kubectl logs <pod-name> -n <namespace>
- 描述Pod详情:
bash复制kubectl describe pod <pod-name> -n <namespace>
- 检查Service端点:
bash复制kubectl get endpoints <service-name>
- 网络连通性测试:
bash复制kubectl run -it --rm debug --image=busybox --restart=Never -- sh
# 在容器内测试服务连通性
wget -qO- http://service-name:port
7.2 Helm特有问题解决
- 模板渲染调试:
bash复制helm template ./chart --debug
- 回滚到上一版本:
bash复制helm rollback <release-name> <revision-number>
- 查看发布历史:
bash复制helm history <release-name>
- 解决依赖问题:
bash复制helm dependency update ./chart
7.3 性能问题诊断
- 查看资源使用情况:
bash复制kubectl top pods -n <namespace>
- 分析CPU profiling(Node.js示例):
bash复制kubectl exec <pod-name> -- node --inspect-brk=0.0.0.0:9229 app.js
- 内存分析(Java示例):
bash复制kubectl exec <pod-name> -- jcmd 1 GC.heap_dump /tmp/heap.hprof
kubectl cp <pod-name>:/tmp/heap.hprof ./heap.hprof
- 网络延迟检测:
bash复制kubectl run -it --rm netshoot --image=nicolaka/netshoot --restart=Never -- bash
# 使用tcpdump、tcptraceroute等工具分析