一 存储卷的概念和类型
为了保证数据的持久性 必须保证
docker容器中的数据存储在容器外部 为了实现数据的持久性存储 在宿主机和容器内做映射 可以保证在容器的生命周期结束 数据依旧可以实现持久性存储 但是在k8s中 由于pod分布在各个不同的节点上 并不能实现不同节点之间持久性数据的共享 并且在节点故障时 可能会导致数据的丢失 为此k8s就引入了外部存储卷的功能
k8s的存储卷类型
[root@master service]# kubectl explain pod.spec.volumes #查看支持的存储类型
KIND: Pod
VERSION: v1
FIELDS:
awsElasticBlockStore <Object>
AWSElasticBlockStore represents an AWS Disk resource that is attached to a
kubelet‘s host machine and then exposed to the pod. More info:
https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
cephfs <Object>
CephFS represents a Ceph FS mount on the host that shares a pod‘s lifetime
cinder <Object>
Cinder represents a cinder volume attached and mounted on kubelets host
machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
configMap <Object>
ConfigMap represents a configMap that should populate this volume
csi <Object>
CSI (Container Storage Interface) represents ephemeral storage that is
handled by certain external CSI drivers (Beta feature).
常用的分类
- emptyDir（临时目录）:Pod删除，数据也会被清除，这种存储成为emptyDir，用于数据的临时存储。多容器数据互通
- hostPath (宿主机目录映射):
- 本地的SAN (iSCSI,FC)、NAS(nfs,cifs,http)存储
- 分布式存储（glusterfs，rbd，cephfs）
- 云存储 （EBS，Azure Disk）
二、emptyDir存储卷演示
一个emptyDir 第一次创建是在一个pod被指定到具体node的时候，并且会一直存在在pod的生命周期当中，正如它的名字一样，它初始化是一个空的目录，pod中的容器都可以读写这个目录，这个目录可以被挂在到各个容器相同或者不相同的的路径下。当一个pod因为任何原因被移除的时候，这些数据会被永久删除。注意：一个容器崩溃了不会导致数据的丢失，因为容器的崩溃并不移除pod.
emptyDir 磁盘的作用：
（1）普通空间，基于磁盘的数据存储
（2）作为从崩溃中恢复的备份点
（3）存储那些那些需要长久保存的数据，例web服务中的数据
默认的，emptyDir 磁盘会存储在主机所使用的媒介上，可能是SSD，或者网络硬盘，这主要取决于你的环境。当然，我们也可以将emptyDir.medium的值设置为Memory来告诉Kubernetes 来挂在一个基于内存的目录tmpfs，因为tmpfs速度会比硬盘块度了，但是，当主机重启的时候所有的数据都会丢失。
[root@k8s-master ~]# kubectl explain pods.spec.volumes.emptyDir #查看emptyDir存储定义
[root@k8s-master ~]# kubectl explain pods.spec.containers.volumeMounts #查看容器挂载方式
[root@k8s-master ~]# mkdir volumes && cd volumes
[root@k8s-master volumes]# vim pod-vol-demo.yaml #创建emptyDir的清单
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
namespace: default
labels:
app: myapp
tier: frontend
annotations:
magedu.com/create-by:"cluster admin"
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
volumeMounts: #在容器内定义挂载存储名称和挂载路径
- name: html
mountPath: /usr/share/nginx/html/
- name: busybox
image: busybox:latest
imagePullPolicy: IfNotPresent
volumeMounts:
- name: html
mountPath: /data/ #在容器内定义挂载存储名称和挂载路径
command: [‘/bin/sh‘,‘-c‘,‘while true;do echo $(date) >> /data/index.html;sleep 2;done‘]
volumes: #定义存储卷
- name: html #定义存储卷名称
emptyDir: {} #定义存储卷类型
[root@master volume]# kubectl apply -f empryDir_pod.yaml
pod/pod-demo created
[root@master volume]# kubectl get pod -l app=myapp
NAME READY STATUS RESTARTS AGE
pod-demo 0/2 ContainerCreating 0 13s
[root@master volume]# kubectl get pod -l app=myapp -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-demo 0/2 ContainerCreating 0 16s <none> node2 <none> <none>
[root@master volume]# kubectl get pod -l app=myapp -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-demo 2/2 Running 0 86s 172.7.104.28 node2 <none> <none>
[root@master volume]# curl 172.7.104.28
Wed Aug 11 09:55:47 UTC 2021
Wed Aug 11 09:55:49 UTC 2021
Wed Aug 11 09:55:51 UTC 2021
Wed Aug 11 09:55:53 UTC 2021
Wed Aug 11 09:55:55 UTC 2021
Wed Aug 11 09:55:57 UTC 2021
Wed Aug 11 09:55:59 UTC 2021
Wed Aug 11 09:56:01 UTC 2021
Wed Aug 11 09:56:03 UTC 2021
Wed Aug 11 09:56:05 UTC 2021
Wed Aug 11 09:56:07 UTC 2021
Wed Aug 11 09:56:09 UTC 2021
Wed Aug 11 09:56:11 UTC 2021
Wed Aug 11 09:56:13 UTC 2021
三、hostPath存储卷
hostPath宿主机路径，就是把pod所在的宿主机之上的脱离pod中的容器名称空间的之外的宿主机的文件系统的某一目录和pod建立关联关系，在pod删除时，存储数据不会丢失。
（1）查看hostPath存储类型定义
[root@k8s-master volumes]# kubectl explain pods.spec.volumes.hostPath
KIND: Pod
VERSION: v1
RESOURCE: hostPath <Object>
DESCRIPTION:
HostPath represents a pre-existing file or directory on the host machine
that is directly exposed to the container. This is generally used for
system agents or other privileged things that are allowed to see the host
machine. Most containers will NOT need this. More info:
https://kubernetes.io/docs/concepts/storage/volumes#hostpath
Represents a host path mapped into a pod. Host path volumes do not support
ownership management or SELinux relabeling.
FIELDS:
path <string> -required- #指定宿主机的路径
Path of the directory on the host. If the path is a symlink, it will follow
the link to the real path. More info:
https://kubernetes.io/docs/concepts/storage/volumes#hostpath
type <string>
Type for HostPath Volume Defaults to "" More info:
https://kubernetes.io/docs/concepts/storage/volumes#hostpath
type：
DirectoryOrCreate 宿主机上不存在创建此目录
Directory 必须存在挂载目录
FileOrCreate 宿主机上不存在挂载文件就创建
File 必须存在文件
(2)清单定义
apiVersion: v1
kind: Pod
metadata:
name: volumes-hostpath-demo
spec:
containers:
- name: filebeat
image: ikubernetes/filebeat:5.6.7-alpine
env:
- name: REDIS_HOST
value: redis.ilinux.io:6379
- name: LOG_LEVEL
value: info
volumeMounts:
- name: varlog
mountPath: /var/log
- name: socket
mountPath: /var/run/docker.sock
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: socket
hostPath:
path: /var/run/docker.sock
[root@master volume]# kubectl get pod
NAME READY STATUS RESTARTS AGE
volumes-hostpath-demo 1/1 Running 0 7m49s
[root@master volume]# kubectl exec -it volumes-hostpath-demo -- bash
OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown
command terminated with exit code 126
[root@master volume]# kubectl exec -it volumes-hostpath-demo -- sh
/ # ls /var/log/
anaconda chrony grubby ntpstats tallylog vmware-vmsvc-root.log
audit containers grubby_prune_debug pods tuned vmware-vmsvc.log
boot.log cron lastlog rhsm vmware-network.1.log vmware-vmtoolsd-root.log
boot.log-20210804 cron-20210808 maillog secure vmware-network.2.log wtmp
boot.log-20210805 dmesg maillog-20210808 secure-20210808 vmware-network.3.log yum.log
btmp dmesg.old messages spooler vmware-network.log
calico firewalld messages-20210808 spooler-20210808 vmware-vgauthsvc.log.0
/ # cat /var/log/yum.log
Apr 19 12:38:24 Installed: 1:bash-completion-2.1-6.el7.noarch
Apr 19 12:38:40 Installed: 2:vim-filesystem-7.4.160-5.el7.x86_64
Apr 19 12:38:42 Installed: 2:vim-common-7.4.160-5.el7.x86_64
Apr 19 12:38:42 Installed: 14:libpcap-1.5.3-11.el7.x86_64
Apr 19 12:38:42 Installed: gpm-libs-1.20.7-5.el7.x86_64
Apr 19 12:38:42 Installed: 2:vim-enhanced-7.4.160-5.el7.x86_64
Apr 19 12:38:43 Installed: 14:tcpdump-4.9.2-3.el7.x86_64
Apr 19 12:38:43 Installed: lsof-4.87-6.el7.x86_64
Apr 19 12:38:43 Installed: lrzsz-0.12.20-36.el7.x86_64
四、nfs共享存储卷演示
nfs使的我们可以挂在已经存在的共享到的我们的Pod中，和emptyDir不同的是，emptyDir会被删除当我们的Pod被删除的时候，但是nfs不会被删除，仅仅是解除挂在状态而已，这就意味着NFS能够允许我们提前对数据进行处理，而且这些数据可以在Pod之间相互传递.并且，nfs可以同时被多个pod挂在并进行读写
（1）在stor01节点上安装nfs，并配置nfs服务
[root@stor01 ~]# yum install -y nfs-utils ==》192.168.56.14
[root@stor01 ~]# mkdir /data/volumes -pv
[root@stor01 ~]# vim /etc/exports
/data/volumes 192.168.56.0/24(rw,no_root_squash)
[root@stor01 ~]# systemctl start nfs
[root@stor01 ~]# showmount -e
Export list for stor01:
/data/volumes 192.168.56.0/24
（2）在node01和node02节点上安装nfs-utils，并测试挂载
[root@k8s-node01 ~]# yum install -y nfs-utils
[root@k8s-node02 ~]# yum install -y nfs-utils
[root@k8s-node02 ~]# mount -t nfs stor01:/data/volumes /mnt
[root@k8s-node02 ~]# mount
......
stor01:/data/volumes on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.56.13,local_lock=none,addr=192.168.56.14)
[root@k8s-node02 ~]# umount /mnt/
（3）创建nfs存储卷的使用清单
[root@k8s-master volumes]# cp pod-hostpath-vol.yaml pod-nfs-vol.yaml
[root@k8s-master volumes]# vim pod-nfs-vol.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-vol-nfs
namespace: default
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
volumes:
- name: html
nfs:
path: /data/volumes
server: stor01
[root@k8s-master volumes]# kubectl apply -f pod-nfs-vol.yaml
pod/pod-vol-nfs created
[root@k8s-master volumes]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
pod-vol-nfs 1/1 Running 0 21s 10.244.2.38 k8s-node02
（3）在nfs服务器上创建index.html
[root@stor01 ~]# cd /data/volumes
[root@stor01 volumes ~]# vim index.html
<h1> nfs stor01</h1>
[root@k8s-master volumes]# curl 10.244.2.38
<h1> nfs stor01</h1>
[root@k8s-master volumes]# kubectl delete -f pod-nfs-vol.yaml #删除nfs相关pod，再重新创建，可以得到数据的持久化存储
pod "pod-vol-nfs" deleted
[root@k8s-master volumes]# kubectl apply -f pod-nfs-vol.yaml