1. K8S基础架构与核心组件解析
Kubernetes(简称K8S)作为容器编排的事实标准,其架构设计遵循了声明式API和控制器模式的核心思想。典型的K8S集群由控制平面(Control Plane)和工作节点(Worker Node)组成,每个部分都包含多个关键组件。
控制平面是集群的大脑,主要包括:
- API Server:集群的唯一切入点,处理所有REST请求
- Scheduler:负责将Pod调度到合适的Node上
- Controller Manager:运行各种控制器(如Deployment、ReplicaSet控制器)
- etcd:分布式键值存储,保存集群所有状态数据
工作节点是实际运行容器的机器,核心组件包括:
- kubelet:节点上的"代理",负责与API Server通信并管理容器
- kube-proxy:维护节点网络规则,实现Service的抽象
- 容器运行时:如Docker、containerd等,实际运行容器
提示:生产环境中建议将控制平面组件部署在至少3个节点上以实现高可用,etcd应采用奇数个节点组成集群。
需要模型API调用? 免费领10W Token,多模型网关一键接入 Claude、DeepSeek 等主流模型。
2. 集群搭建实战:从零部署K8S环境
2.1 环境准备与前置条件
在Ubuntu 22.04 LTS上部署K8S集群前,需确保:
- 每个节点至少2GB内存和2核CPU
- 禁用swap分区(sudo swapoff -a并注释/etc/fstab中的swap行)
- 各节点间网络互通且主机名解析正常
- 安装容器运行时(推荐containerd)
配置系统内核参数:
bash复制cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
# 设置必需的sysctl参数
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
2.2 使用kubeadm快速部署
安装kubeadm、kubelet和kubectl:
bash复制sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
初始化控制平面(在主节点执行):
bash复制sudo kubeadm init --pod-network-cidr=10.244.0.0/16
按照命令输出提示配置kubectl:
bash复制mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
安装网络插件(以Flannel为例):
bash复制kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
将工作节点加入集群(在各工作节点执行):
bash复制kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>
3. 核心资源对象详解与操作
3.1 Pod:K8S的最小调度单元
Pod是K8S中最小的部署单元,每个Pod包含一个或多个紧密关联的容器。创建nginx Pod示例:
yaml复制apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.25
ports:
- containerPort: 80
应用配置:
bash复制kubectl apply -f nginx-pod.yaml
查看Pod状态:
bash复制kubectl get pods -o wide
kubectl describe pod nginx-pod
注意:直接创建Pod在实际生产中很少使用,通常通过Deployment等更高级资源来管理。
3.2 Deployment:声明式管理应用
Deployment提供了Pod的声明式更新能力,支持滚动更新和回滚。创建Deployment示例:
yaml复制apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.25
ports:
- containerPort: 80
常用操作:
bash复制# 查看部署状态
kubectl get deployments
# 查看关联的ReplicaSet
kubectl get rs
# 触发滚动更新(修改镜像版本)
kubectl set image deployment/nginx-deployment nginx=nginx:1.26
# 查看更新状态
kubectl rollout status deployment/nginx-deployment
# 回滚到上一版本
kubectl rollout undo deployment/nginx-deployment
3.3 Service:暴露应用服务
Service为Pod提供稳定的访问入口,主要类型包括:
- ClusterIP:默认类型,集群内部访问
- NodePort:通过节点端口暴露服务
- LoadBalancer:使用云提供商的负载均衡器
创建NodePort Service示例:
yaml复制apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30080
验证服务:
bash复制kubectl get svc
curl http://<node-ip>:30080
4. 高级特性与生产实践
4.1 ConfigMap与Secret管理配置
ConfigMap用于存储非敏感配置数据:
yaml复制apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
log_level: "info"
database_url: "mysql://db:3306"
Secret用于存储敏感信息(自动base64编码):
bash复制echo -n 'admin' | base64
echo -n 'password123' | base64
yaml复制apiVersion: v1
kind: Secret
metadata:
name: db-secret
type: Opaque
data:
username: YWRtaW4=
password: cGFzc3dvcmQxMjM=
在Pod中使用:
yaml复制env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: db-secret
key: username
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: app-config
key: log_level
4.2 资源配额与限制
为命名空间设置资源配额:
yaml复制apiVersion: v1
kind: ResourceQuota
metadata:
name: mem-cpu-quota
spec:
hard:
requests.cpu: "2"
requests.memory: 4Gi
limits.cpu: "4"
limits.memory: 8Gi
在Pod中指定资源请求和限制:
yaml复制resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
4.3 使用Horizontal Pod Autoscaler自动扩缩容
创建HPA自动根据CPU使用率调整Pod数量:
bash复制kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=2 --max=10
或通过YAML定义:
yaml复制apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: nginx-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: nginx-deployment
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
查看HPA状态:
bash复制kubectl get hpa
kubectl describe hpa nginx-hpa
5. 监控与日志管理
5.1 部署Prometheus监控集群
使用Helm安装Prometheus Stack:
bash复制helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/kube-prometheus-stack
访问Grafana(默认用户名admin,密码prom-operator):
bash复制kubectl port-forward svc/prometheus-grafana 3000:80
5.2 配置应用日志收集
建议的日志收集方案:
- 使用Sidecar容器将日志输出到stdout/stderr
- 部署DaemonSet形式的日志收集器(如Fluentd)
- 将日志发送到集中式存储(如Elasticsearch)
示例Fluentd配置:
yaml复制apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd
namespace: kube-system
spec:
selector:
matchLabels:
name: fluentd
template:
metadata:
labels:
name: fluentd
spec:
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch8-1
env:
- name: FLUENT_ELASTICSEARCH_HOST
value: "elasticsearch-logging"
- name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
6. 生产环境最佳实践
6.1 多租户与命名空间隔离
创建开发和生产命名空间:
bash复制kubectl create namespace dev
kubectl create namespace prod
设置资源配额(见4.2节)
使用NetworkPolicy实现网络隔离:
yaml复制apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: dev
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
6.2 安全加固措施
- 启用Pod安全准入控制:
bash复制kube-apiserver --enable-admission-plugins=PodSecurity
- 使用SecurityContext限制容器权限:
yaml复制securityContext:
runAsNonRoot: true
runAsUser: 1000
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
- 定期轮换证书:
bash复制kubeadm certs renew all
6.3 备份与灾难恢复
- 定期备份etcd数据:
bash复制ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
snapshot save /backup/etcd-snapshot-$(date +%Y%m%d).db
- 使用Velero进行集群资源备份:
bash复制velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.7.0 \
--bucket my-backup-bucket \
--secret-file ./credentials-velero \
--use-volume-snapshots=false \
--backup-location-config region=us-west-2
- 创建定期备份计划:
bash复制velero schedule create daily-backup \
--schedule="0 1 * * *" \
--include-namespaces=prod \
--ttl 168h
在实际生产部署中,我发现合理设置Pod反亲和性可以显著提高应用可用性。例如,对于有3个副本的Deployment,可以添加如下配置确保Pod分散在不同节点:
yaml复制affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx
topologyKey: kubernetes.io/hostname
另一个常见问题是资源限制设置不当导致OOMKill。建议:
- 为所有容器设置合理的内存限制
- 监控容器的实际内存使用情况
- 为Java等有堆内存管理的应用设置适当的JVM参数
