在StatefulSet中使用LocalVolume存储卷保持节点一致

StatefulSet是一种有状态服务,其存储卷的使用有多种方式:

使用共享存储,这时在模板中定义一个volume卷,可以给多个pod共享;

每个pod配置独立的存储卷,使用非共享存储(块存储)时需要这样配置,通过配置volumeClaimTemplates实现;

对于StatefulSet使用localvolume的场景,和上述两种情况都不一样:

通过volumeClaimTemplates自动生成localvolume的controller没有提供;
localvolume本身定义了节点信息,会导致使用这个pv的pod调度到相应节点;
StatefulSet如果多个pod使用同一个localvolume,会导致多个pod调度到相同节点;

本文先通过手动创建pvc、localvolume的方式,并利用statefulset 自动生成pvc的规则,在statefulset中配置localvolume;

场景:

生成一个StatefulSet应用,每个pod定义到特定节点;StatefulSet中定义2个Pod,每个pod使用两个pvc,共需要创建4个pvc和4个localvolume;

如下图所示:

在StatefulSet中使用LocalVolume存储卷保持节点一致

模板:

下面分别给出4个pv、pvc的模板,4个pv分别配置到cn-shenzhen.i-wz9gvy73m4qyk03xzg1y、cn-shenzhen.i-wz9c9m0m4oldr6mt89rd两个节点上;

pv1, pvc1:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-volume-a-1
  labels:
    alicloud-pvname: local-volume-a-1
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-volume
  local:
    path: /local1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - cn-shenzhen.i-wz9c9m0m4oldr6mt89rd
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-volume-a-web-0
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: local-volume
  selector:
    matchLabels:
      alicloud-pvname: local-volume-a-1

pv2, pvc2:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-volume-a-2
  labels:
    alicloud-pvname: local-volume-a-2
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-volume
  local:
    path: /local1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - cn-shenzhen.i-wz9gvy73m4qyk03xzg1y
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-volume-a-web-1
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: local-volume
  selector:
    matchLabels:
      alicloud-pvname: local-volume-a-2

pv3, pvc3:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-volume-b-1
  labels:
    alicloud-pvname: local-volume-b-1
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-volume
  local:
    path: /local2
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - cn-shenzhen.i-wz9c9m0m4oldr6mt89rd
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-volume-b-web-0
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: local-volume
  selector:
    matchLabels:
      alicloud-pvname: local-volume-b-1

pv4, pvc4:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-volume-b-2
  labels:
    alicloud-pvname: local-volume-b-2
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-volume
  local:
    path: /local2
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - cn-shenzhen.i-wz9gvy73m4qyk03xzg1y
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: local-volume-b-web-1
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: local-volume
  selector:
    matchLabels:
      alicloud-pvname: local-volume-b-2

创建上述pv pvc后,创建应用:

创建应用:

创建的时候注意pvc的名字的构成:
pvc的名字 = statefulset-name + volume-name + 序号

local-volume-a-web-0 = "local-volume-a" + "-" + "web" + "-" + "0"

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: local-volume-a
          mountPath: /data1
        - name: local-volume-b
          mountPath: /data2
  volumeClaimTemplates:
  - metadata:
      name: local-volume-a
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-volume"
      resources:
        requests:
          storage: 20Gi
  - metadata:
      name: local-volume-b
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-volume"
      resources:
        requests:
          storage: 20Gi

验证:

# kubectl get pod | grep web
web-0                              1/1     Running     0          15m
web-1                              1/1     Running     0          14m
# kubectl get pvc | grep local
local-volume-a-web-0                       Bound    local-volume-a-1         20Gi       RWO            local-volume        15m
local-volume-a-web-1                       Bound    local-volume-a-2         20Gi       RWO            local-volume        15m
local-volume-b-web-0                       Bound    local-volume-b-1         20Gi       RWO            local-volume        15m
local-volume-b-web-1                       Bound    local-volume-b-2         20Gi       RWO            local-volume        15m
# kubectl get pv | grep local
local-volume-a-1         20Gi       RWO            Retain           Bound      default/local-volume-a-web-0                       local-volume                 15m
local-volume-a-2         20Gi       RWO            Retain           Bound      default/local-volume-a-web-1                       local-volume                 17m
local-volume-b-1         20Gi       RWO            Retain           Bound      default/local-volume-b-web-0                       local-volume                 15m
local-volume-b-2         20Gi       RWO            Retain           Bound      default/local-volume-b-web-1                       local-volume                 17m

删除pod,验证新建pod调度到相同节点:

# kubectl describe pod web-0 | grep Node
Node:               cn-shenzhen.i-wz9c9m0m4oldr6mt89rd/192.168.0.12
Node-Selectors:  <none>
# kubectl delete pod web-0
pod "web-0" deleted
# kubectl get pod
NAME                               READY   STATUS      RESTARTS   AGE
web-0                              1/1     Running     0          10s
web-1                              1/1     Running     0          15m
# kubectl describe pod web-0 | grep Node
Node:               cn-shenzhen.i-wz9c9m0m4oldr6mt89rd/192.168.0.12
Node-Selectors:  <none>
上一篇:入门视频采集与处理(学会分析YUV数据)


下一篇:名词模块_运动与健康