一、nfs存储卷
kubernetes的NFS存储卷用于将某实现存在的NFS服务器上导出export的存储空间挂在到pod中以供容器使用
与emptyDir不同的是、NFS存储卷在POD对象终止后仅是被卸载而非删除、另外NFS是文件系统级共享服务,它支持同时存在的多路关在请求
1、字段详解
[root@master ~]# kubectl explain pod.spec.volumes.nfs KIND: Pod VERSION: v1 RESOURCE: nfs <Object> DESCRIPTION: NFS represents an NFS mount on the host that shares a pod's lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs Represents an NFS mount that lasts the lifetime of a pod. NFS volumes do not support ownership management or SELinux relabeling. FIELDS: path <string> -required- #NFS服务器导出共享的文件系统路径、必选字段 Path that is exported by the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs readOnly <boolean> #是以只读方式挂在、默认为false ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs server <string> -required- #NFS服务器的ip地址、默认是false Server is the hostname or IP address of the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
2、配置清单
[root@master chapter7]# cat vol-nfs1.yaml apiVersion: v1 kind: Pod metadata: name: vol-nfs-pod spec: containers: - name: myapp image: ikubernetes/myapp:v1 volumeMounts: - name: html mountPath: /usr/share/nginx/html volumes: - name: html nfs: path: /data/volumes server: node2
3、nfs server搭建
yum install nfs-utils rpcbind -y #所有节点均安装 [root@node2 volumes]# cat /etc/exports /data/volumes 192.168.118.0/24 (rw,no_root_squash) # 启动rpcbind服务 systemctl restart rpcbind #启动 systemctl restart nfs-server [root@node2 volumes]# showmount -e Export list for node2: /data/volumes (everyone)
4、挂载主页
[root@node2 volumes]# ls index.html [root@node2 volumes]# pwd /data/volumes [root@node2 volumes]# cat index.html <h1>NFS node2</h1>
5、创建运行vol-nfs.yaml
[root@master chapter7]# kubectl get pods NAME READY STATUS RESTARTS AGE pod-example 1/1 Running 1 9d vol-emptydir-pod 2/2 Running 0 5d1h vol-hostpath-pod 1/1 Running 0 4d22h [root@master chapter7]# kubectl apply -f vol-nfs1.yaml pod/vol-nfs-pod created [root@master chapter7]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-example 1/1 Running 1 9d 10.244.2.9 node2 <none> <none> vol-emptydir-pod 2/2 Running 0 5d1h 10.244.2.68 node2 <none> <none> vol-hostpath-pod 1/1 Running 0 4d22h 10.244.1.43 node1 <none> <none> vol-nfs-pod 1/1 Running 0 4m24s 10.244.1.50 node1 <none> <none> [root@master chapter7]# curl 10.244.1.50 <h1>NFS node2</h1>
6、删除pod数据依然存在
为了测试其数据持久化效果,下面删除pod资源vol-nfs-pod、并重建后检测数据是否依然能访问
[root@master chapter7]# kubectl delete -f vol-nfs1.yaml pod "vol-nfs-pod" deleted [root@master chapter7]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-example 1/1 Running 1 9d 10.244.2.9 node2 <none> <none> vol-emptydir-pod 2/2 Running 0 5d1h 10.244.2.68 node2 <none> <none> vol-hostpath-pod 1/1 Running 0 4d22h 10.244.1.43 node1 <none> <none> [root@master chapter7]# kubectl apply -f vol-nfs1.yaml pod/vol-nfs-pod created [root@master chapter7]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-example 1/1 Running 1 9d 10.244.2.9 node2 <none> <none> vol-emptydir-pod 2/2 Running 0 5d1h 10.244.2.68 node2 <none> <none> vol-hostpath-pod 1/1 Running 0 4d22h 10.244.1.43 node1 <none> <none> vol-nfs-pod 1/1 Running 0 6s 10.244.1.51 node1 <none> <none> [root@master chapter7]# curl 10.244.1.51 <h1>NFS node2</h1>
这里应确保实现要存在一个名为nfs.ilinux.io的NFS服务器、其输出了/data/redis目录,并授权给了kubernetes集群中的节点访问、主机和目录都可以按需进行调整
从上面的命令中可以看出,此前创建的键mykey及其数据在pod资源重建后依然存在、这表明再删除pod资源时、其关联的外部存储卷并不会被一同删除、如果需要清除此类的数据、需要用户通过存储系统的管理接口手动进行
二、RBD存储卷
1、配置pod资源使用RBD存储卷、满足条件
- 存在某可用的Ceph RBD存储集群,否则就需要创建一个
- 在Ceph集群中创建一个能满足Pod资源数据存储需要的存储影响(images)
- 在kubernetes集群内的各节点上安装Ceph客户端程序包(ceph-common)
2、字段详解
在配置RBD类型的存储卷时、需要制定要连接的目标服务器和认证信息等、这一点通常使用一下嵌套字段进行定义
[root@master ~]# kubectl explain pod.spec.volumes.rbd KIND: Pod VERSION: v1 RESOURCE: rbd <Object> DESCRIPTION: RBD represents a Rados Block Device mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md Represents a Rados Block Device mount that lasts the lifetime of a pod. RBD volumes support ownership management and SELinux relabeling. FIELDS: fsType <string> #要挂在的存储卷的文件系统类型,至少应该是节点操作系统支持的文件系统,如:ext4", "xfs", "ntfs".默认为"ext4" Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd image <string> -required- #rados image的名称,必选字段 The rados image name. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it keyring <string> #RBD用户认证时的keyring文件路径,默认为/etc/ceph/keyring Keyring is the path to key ring for RBDUser. Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it monitors <[]string> -required- #ceph存储见识其,逗号分隔的字符串列表;必选字段 A collection of Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it pool <string> #rados 存储池名称,默认为RBD The rados pool name. Default is rbd. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it readOnly <boolean> #是否以只读的方式进行访问 ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it secretRef <Object> #RBD用户认证时使用的保存有相应认真信息的secret对象,会覆盖由keyring字段提供的密钥信息 SecretRef is name of the authentication secret for RBDUser. If provided overrides keyring. Default is nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it user <string> #rados 用户名,默认为admin The rados user name. Default is admin. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
3、实验架构图
此示例依赖于实现存在的一个ceph存储集群
这里假设其监视器的地址为:172.16.0.56、172.16.0.57、172.16.0.58三个主机IP
并且集群上存储池kube中存在创建好的应先redis、此影响用油ext4文件系统
ceph客户端访问集群时需要实现完成认真之后才能进行后续的访问操作、此示例上、其认真信息保存于名为Ceph-secret的secret资源对象中
4、示例模板
[root@master chapter7]# cat vol-rbd.yaml apiVersion: v1 kind: Pod metadata: name: vol-rbd-pod spec: containers: - name: redis image: redis:4-alpine ports: - containerPort: 6379 name: redisport volumeMounts: - mountPath: /data name: redis-rbd-vol volumes: - name: redis-rbd-vol rbd: monitors: - '172.16.0.56:6789' - '172.16.0.57:6789' - '172.16.0.58:6789' pool: kube image: redis fsType: ext4 readOnly: false user: admin secretRef: name: ceph-secret
三、GlusterFS存储卷
要配置pod资源使用GlusterFS存储卷、需要事先满足一下前提条件
1、前提条件
1、存储在某可用的GlusterFS存储集群、否则就要创建一个
2、在GlusterFS集群中创建一个能满足pod资源数据存储需要的卷
3、在kubernetes集群内的各节点上安装GlusterFS客户端程序包(glusterf和gluseterfs-fuse)
4、另外、若要基于GlusterFS是哦用存储卷的动态供给机制、还需要实现部署heketi,它用于为GlusterFS集群提供RESTFUL风格的管理接口
2、字段属性详解
[root@master ~]# kubectl explain pod.spec.volumes.glusterfs KIND: Pod VERSION: v1 RESOURCE: glusterfs <Object> DESCRIPTION: Glusterfs represents a Glusterfs mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md Represents a Glusterfs mount that lasts the lifetime of a pod. Glusterfs volumes do not support ownership management or SELinux relabeling. FIELDS: endpoints <string> -required- #Endpoints 资源的名称、此资源需要实现存在,用于提供Cluster集群的部分节点信息作为其访问入口;必选字段 EndpointsName is the endpoint name that details Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod path <string> -required- #用到的glusterfs集群的卷路径、如:kube-redis;必选字段 Path is the Glusterfs volume path. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod readOnly <boolean> #是否为只读卷 ReadOnly here will force the Glusterfs volume to be mounted with read-only permissions. Defaults to false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod
3、资源清单
[root@master chapter7]# cat vol-glusterfs.yaml apiVersion: v1 kind: Pod metadata: name: vol-glusterfs-pod labels: app: redis spec: containers: - name: redis image: redis:alpine ports: - containerPort: 6379 name: redisport volumeMounts: - mountPath: /data name: redisdata volumes: - name: redisdata glusterfs: endpoints: glusterfs-endpoints path: kube-redis readOnly: false
vol-glusterfs.yaml 它使用了glusterfs存储卷持久保存应用数据、它通过glusterfs-endpolints资源中定义的glusterfs集群节点信息接入集群
并以kube-redis卷作为pod资源的存储卷。glusterfs-endpolints资源需要在kubernetes集群中事先创建、而kube-redis则需要事先创建Gluster集群
4、创建运行
用于访问cluster集群的相关结点信息要实现保存于某特定的endpolints资源中、例如上面示例中调用的glusterfs-endpolints、此类的endpolints
资源可由用户根据实际需要手动创建、例如、下面的保存于glusterfs-endpolints.yaml文件中的资源示例中定义了三个接入相关的gluster存储集群
的节点gfs01.ilinux.io、gfs01.ilinux.io和gfs03.ilinux.io、期中的端口信息仅为满足endpolints资源必选字段要求、因此其值可以随意填写
[root@master chapter7]# cat glusterfs-endpoints.yaml apiVersion: v1 kind: Endpoints metadata: name: glusterfs-endpoints subsets: - addresses: - ip: 172.16.2.36 ports: - port: 24007 name: glusterd - addresses: - ip: 172.16.2.37 ports: - port: 24007 name: glusterd - addresses: - ip: 172.16.2.38 ports: - port: 24007 name: glusterd
首先创建endpolints资源glusterfs-endpoints,然后再创建pod资源vol-glusterfs即可测试其数据持久化存储的效果
四、cinder存储卷
1、字段详解
openstack构建的iaas环境中时、cinder的块存储功能可为pod资源提供外部持久存储的幼小方式
在pod资源上定义使用cinder存储卷时,其可用的嵌套字段包含如下几个
[root@master ~]# kubectl explain pod.spec.volumes.cinder KIND: Pod VERSION: v1 RESOURCE: cinder <Object> DESCRIPTION: Cinder represents a cinder volume attached and mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md Represents a cinder volume resource in Openstack. A Cinder volume must exist before mounting to a container. The volume must also be in the same region as the kubelet. Cinder volumes support ownership management and SELinux relabeling. FIELDS: fsType <string> #要挂在的存储卷的问价那系统类型,至少应该是接待能操作系统支持的问价那系统 Filesystem type to mount. Must be a filesystem type supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md readOnly <boolean> #是否以只读方式访问 Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/mysql-cinder-pd/README.md secretRef <Object> Optional: points to a secret object containing parameters used to connect to OpenStack. volumeID <string> -required- #用于表示cinder中的存储卷的卷标识符、必选字段 volume id used to identify the volume in cinder. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
2、模板示例
下面的资源清单是定义在vol-cinder.yaml 文件中的使用示例、假设在openstack环境中有创建好的cinder卷可用
[root@master chapter7]# cat vol-cinder.yaml apiVersion: v1 kind: Pod metadata: name: vol-cinder-pod spec: containers: - image: mysql name: mysql args: - "--ignore-db-dir" - "lost+found" env: - name: MYSQL_ROOT_PASSWORD value: YOUR_PASS ports: - containerPort: 3306 name: mysqlport volumeMounts: - name: mysqldata mountPath: /var/lib/mysql volumes: - name: mysqldata cinder: volumeID: e2b8d2f7-wece-90d1-a505-4acf607a90bc fsType: ext4