failed to create kubelet: misconfiguration: kubelet cgroup driver: “systemd“ is different from docke

在k8s集群中调度pod时,发现调度到其中一个节点的pod都是pending状态,怀疑该节点有问题。通过kubectl get node -o wide发现该节点为notReady状态。
在该问题节点上执行命令journalctl -f -u kubelet查看日志状态

[root@k8s-node02 docker.service.d]# journalctl -f -u kubelet
-- Logs begin at 二 2020-09-01 00:02:17 CST. --
12月 13 12:31:40 k8s-node02 kubelet[22475]: I1213 12:31:40.943056   22475 client.go:75] Connecting to docker on unix:///var/run/docker.sock
12月 13 12:31:40 k8s-node02 kubelet[22475]: I1213 12:31:40.943098   22475 client.go:104] Start docker client with request timeout=2m0s
12月 13 12:31:40 k8s-node02 kubelet[22475]: W1213 12:31:40.958727   22475 docker_service.go:561] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
12月 13 12:31:40 k8s-node02 kubelet[22475]: I1213 12:31:40.958757   22475 docker_service.go:238] Hairpin mode set to "hairpin-veth"
12月 13 12:31:40 k8s-node02 kubelet[22475]: I1213 12:31:40.960800   22475 docker_service.go:253] Docker cri networking managed by cni
12月 13 12:31:40 k8s-node02 kubelet[22475]: I1213 12:31:40.970314   22475 docker_service.go:258] Docker Info: &{ID:LQ2Y:BCW5:SPKQ:67DS:VLFK:6KRX:ZYIN:2EB6:MWXS:3VZM:YONB:R3JK Containers:31 ContainersRunning:15 ContainersPaused:0 ContainersStopped:16 Images:23 Driver:overlay2 DriverStatus:[[Backing Filesystem xfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:[] Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:36 SystemTime:2020-12-13T12:31:40.96153653+08:00 LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.4.224-1.el7.elrepo.x86_64 OperatingSystem:CentOS Linux 7 (Core) OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc0007040e0 NCPU:2 MemTotal:2079207424 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:k8s-node02 Labels:[] ExperimentalBuild:false ServerVersion:19.03.9 ClusterStore: ClusterAdvertise: Runtimes:map[runc:{Path:runc Args:[]}] DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:[] Nodes:0 Managers:0 Cluster:<nil> Warnings:[]} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7ad184331fa3e55e52b890ea95e65ba581ae3429 Expected:7ad184331fa3e55e52b890ea95e65ba581ae3429} RuncCommit:{ID:dc9208a3303feef5b3839f4323d9beb36df0a9dd Expected:dc9208a3303feef5b3839f4323d9beb36df0a9dd} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnings:[]}
12月 13 12:31:40 k8s-node02 kubelet[22475]: F1213 12:31:40.970424   22475 server.go:273] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
12月 13 12:31:40 k8s-node02 systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
12月 13 12:31:40 k8s-node02 systemd[1]: Unit kubelet.service entered failed state.
12月 13 12:31:40 k8s-node02 systemd[1]: kubelet.service failed.

其中错误原因描述:failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs",发现设置的docker设置cgroup名字与kubelete的不一致,修改/etc/docker/daemon.json配置如下

{
        "exec-opts": ["native.cgroupdriver=systemd"],
        "log-driver": "json-file",
        "log-opts": {
                "max-size": "100m"
        }
}

再用journalctl -f -u kubelet命令查看节点状态,发现节点正常,Ready状态。

上一篇:Last-Modified和ETag以及Apache和Nginx中的配置


下一篇:Pod status 状态解释