K8S版本:1.17.11
今天查看kubelet日志的时候,发信一堆报错:
查看kubelet日志:
]# journalctl -f -u kubelet
]# tail -f /var/log/messages
3986 Dec 2 16:40:41 master kubelet: E1202 16:40:41.294907 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3987 Dec 2 16:40:51 master kubelet: E1202 16:40:51.357545 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3988 Dec 2 16:41:01 master kubelet: E1202 16:41:01.387170 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3989 Dec 2 16:41:11 master kubelet: E1202 16:41:11.413174 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3990 Dec 2 16:41:21 master kubelet: E1202 16:41:21.479766 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3991 Dec 2 16:41:31 master kubelet: E1202 16:41:31.551083 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3992 Dec 2 16:41:41 master kubelet: E1202 16:41:41.595900 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3993 Dec 2 16:41:51 master kubelet: E1202 16:41:51.662374 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3994 Dec 2 16:42:01 master kubelet: E1202 16:42:01.716067 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3995 Dec 2 16:42:11 master kubelet: E1202 16:42:11.773051 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3996 Dec 2 16:42:21 master kubelet: E1202 16:42:21.826902 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3997 Dec 2 16:42:31 master kubelet: E1202 16:42:31.876961 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3998 Dec 2 16:42:41 master kubelet: E1202 16:42:41.932225 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service" 3999 Dec 2 16:42:52 master kubelet: E1202 16:42:52.034354 1010 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get c ontainer info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
网上搜了一堆,大致原因是kubernetes和docker版本兼容性问题,查看docker版本
]# docker info Client: Debug Mode: false Server: Containers: 34 Running: 26 Paused: 0 Stopped: 8 Images: 77 Server Version: 19.03.13 Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs
网上方法:
在10-kubeadm.conf文件中"KUBELET_CGROUP_ARGS"属性添加"--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice" ,如果不存在"KUBELET_CGROUP_ARGS"属性则新增。
10-kubeadm.conf的路径网上提到的多为:/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
而我的K8S尝试了之后并不好使(应该是我的版本太低),并且kubelet启动失败
找到一篇文章:https://*.com/questions/46726216/kubelet-fails-to-get-cgroup-stats-for-docker-and-kubelet-services
按照以下方法修改后报错消失
]# vim /etc/sysconfig/kubelet KUBELET_EXTRA_ARGS="--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice"
重新加载配置文件
systemctl daemon-reload systemctl restart kubelet