1. Ingress-Nginx 核心概念解析
在Kubernetes集群中,服务暴露和流量管理一直是运维人员面临的核心挑战。传统做法是通过NodePort或LoadBalancer类型的Service来暴露服务,但这些方式在复杂路由、TLS终止等场景下显得力不从心。这正是Ingress-Nginx发挥作用的地方。
1.1 Ingress 的本质与价值
Ingress不是具体的服务,而是Kubernetes中的一种API对象,它定义了从集群外部到内部服务的HTTP/HTTPS路由规则。可以把Ingress理解为集群的"智能路由网关",它基于以下要素进行流量分发:
- 主机名(如api.example.com)
- URL路径(如/v1/users)
- 请求头等特征
与常规Service相比,Ingress的核心优势在于:
- 精细化路由:支持基于域名和路径的多服务共享同一IP
- 集中式TLS管理:可以在Ingress层面统一处理HTTPS证书
- 七层流量特征:支持基于HTTP协议的复杂路由规则
1.2 Ingress控制器的工作机制
Ingress资源本身不会产生任何效果,必须配合Ingress控制器才能工作。控制器的主要职责包括:
- 持续监控Ingress资源的变化
- 动态生成Nginx配置
- 热加载Nginx服务
- 处理证书生命周期
当创建一个Ingress资源时,控制器会:
bash复制1. 监听API Server的Ingress变更事件
2. 校验规则有效性
3. 转换为Nginx配置片段
4. 合并到主配置文件
5. 执行nginx -s reload
1.3 Ingress-Nginx的架构特点
官方Ingress-Nginx控制器采用以下架构设计:
- 控制循环:使用client-go库监听Kubernetes API
- Nginx进程:运行在Pod中的主工作进程
- Lua扩展:通过OpenResty增强功能
- 动态配置:通过共享内存实现配置热更新
数据平面处理流量的典型路径:
code复制外部请求 → 节点端口 → Service iptables → Nginx Pod → 后端Pod
2. 生产级部署方案详解
2.1 Helm部署最佳实践
2.1.1 定制化配置要点
修改values.yaml时需要特别关注以下参数:
yaml复制controller:
replicaCount: 3 # 根据集群规模调整
hostNetwork: false # 云环境建议关闭
dnsPolicy: ClusterFirst
service:
type: LoadBalancer # 云环境使用
externalTrafficPolicy: Local # 保持源IP
resources:
requests:
cpu: 1000m
memory: 1Gi
affinity: # 反亲和性配置
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values: [ingress-nginx]
topologyKey: kubernetes.io/hostname
2.1.2 安装流程优化
推荐使用以下安装流程:
bash复制# 添加认证信息(私有仓库场景)
helm registry login registry.example.com
# 定制安装
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
--version 4.11.1 \
--namespace ingress-nginx \
--create-namespace \
--values custom-values.yaml \
--set controller.image.digest="sha256:e6439a12b520..." \
--atomic --wait
关键参数说明:
--atomic:失败时自动回滚--wait:等待所有资源就绪--timeout:适当延长超时时间(默认5分钟可能不够)
2.2 裸YAML部署方案
2.2.1 关键资源配置解析
- RBAC配置:
yaml复制apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: ingress-nginx
rules:
- apiGroups: [""]
resources: ["configmaps", "endpoints", "nodes", "pods", "secrets"]
verbs: ["list", "watch"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "watch"]
- 服务暴露配置:
yaml复制apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-controller
spec:
ports:
- name: http
port: 80
targetPort: 80
nodePort: 30080
- name: https
port: 443
targetPort: 443
nodePort: 30443
selector:
app.kubernetes.io/component: controller
type: NodePort
2.2.2 部署后验证步骤
- 检查Pod状态:
bash复制kubectl get pods -n ingress-nginx -l app.kubernetes.io/component=controller \
-o jsonpath='{range .items[*]}{.status.phase}{"\t"}{.status.podIP}{"\n"}{end}'
- 检查服务端点:
bash复制kubectl get endpoints -n ingress-nginx ingress-nginx-controller
- 测试Nginx配置:
bash复制kubectl exec -n ingress-nginx ingress-nginx-controller-xxx -- \
nginx -T | grep "server_name"
3. 高级配置与优化策略
3.1 性能调优指南
3.1.1 Nginx参数优化
在ConfigMap中添加调优参数:
yaml复制data:
worker-processes: "4"
worker-connections: "65536"
keep-alive-requests: "1000"
upstream-keepalive-connections: "200"
proxy-buffer-size: "16k"
proxy-buffers-number: "4"
3.1.2 内核参数调整
在Pod的initContainer中添加:
yaml复制initContainers:
- name: sysctl
image: busybox
securityContext:
privileged: true
command:
- /bin/sh
- -c
- |
sysctl -w net.core.somaxconn=65535
sysctl -w net.ipv4.tcp_max_syn_backlog=8192
3.2 安全加固方案
3.2.1 TLS最佳实践
- 创建证书Secret:
bash复制kubectl create secret tls example-tls \
--cert=fullchain.pem \
--key=privkey.pem \
--namespace=default
- Ingress配置示例:
yaml复制spec:
tls:
- hosts:
- api.example.com
secretName: example-tls
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-service
port:
number: 80
3.2.2 网络策略限制
yaml复制apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ingress-nginx-isolation
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
policyTypes:
- Ingress
ingress:
- ports:
- protocol: TCP
port: 80
- protocol: TCP
port: 443
from:
- namespaceSelector: {}
4. 实战案例:多服务路由配置
4.1 路径重写场景
yaml复制apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: path-rewrite
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /api(/|$)(.*)
pathType: Prefix
backend:
service:
name: api-service
port:
number: 8080
4.2 灰度发布配置
yaml复制apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: canary-release
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "20"
spec:
rules:
- host: app.example.com
http:
paths:
- path: /
backend:
service:
name: app-v2
port:
number: 80
5. 运维监控与排错
5.1 监控指标采集
Prometheus配置示例:
yaml复制scrape_configs:
- job_name: 'ingress-nginx'
metrics_path: '/metrics'
static_configs:
- targets: ['ingress-nginx-controller.ingress-nginx.svc.cluster.local:10254']
关键监控指标:
nginx_ingress_controller_requests:请求量统计nginx_ingress_controller_request_duration_seconds:延迟分布nginx_ingress_controller_nginx_process_connections:连接数
5.2 常见问题排查
5.2.1 502错误排查流程
- 检查后端服务健康状态:
bash复制kubectl get endpoints <service-name>
- 检查Nginx upstream配置:
bash复制kubectl exec -n ingress-nginx ingress-nginx-controller-xxx -- \
cat /etc/nginx/nginx.conf | grep -A10 "<service-name>"
- 检查连接日志:
bash复制kubectl logs -n ingress-nginx ingress-nginx-controller-xxx --tail=100 | grep "502"
5.2.2 配置不生效排查
- 检查Ingress资源状态:
bash复制kubectl describe ingress <ingress-name>
- 查看控制器事件:
bash复制kubectl get events -n ingress-nginx --field-selector involvedObject.kind=Ingress
- 检查Nginx配置热加载:
bash复制kubectl exec -n ingress-nginx ingress-nginx-controller-xxx -- \
nginx -s reload && echo $?
6. 版本升级与维护
6.1 滚动升级策略
- 检查版本兼容性:
bash复制kubectl get ingressclasses -o jsonpath='{.items[*].controller}' | sort -u
- 分阶段升级流程:
bash复制# 先升级Canary实例
helm upgrade ingress-nginx-canary ingress-nginx/ingress-nginx \
--version 4.12.0 \
--namespace ingress-nginx-canary
# 验证稳定后升级生产实例
helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
--version 4.12.0 \
--namespace ingress-nginx \
--atomic --wait
6.2 长期维护建议
- 定期检查废弃API:
bash复制kubectl get ingress -o json | jq '.items[].apiVersion' | sort -u
- 资源清理脚本:
bash复制# 清理无效Ingress资源
kubectl get ingress --all-namespaces -o json | \
jq -r '.items[] | select(.metadata.annotations["kubernetes.io/ingress.class"] != "nginx") | .metadata.name'
- 配置备份方案:
bash复制# 备份关键配置
kubectl get cm -n ingress-nginx ingress-nginx-controller -o yaml > backup.yaml