docker及K8S快速安装

2023-12-09 21:24:03

#建议：先全部在一台master上执行完成集群的安装，最后node节点再加入集群

#硬件环境的要求： cpu：2c memory：4G/2G #软件环境的要求：三天

docker环境准备

准备3台虚拟机（centos-8.2）并运行以下脚本：

新建脚本文件

#!/bin/bash

#解决依赖关系
yum install -y yum-utils zlib zlib-devel openssl openssl-devel pcre pcre-devel gcc gcc-c++ autoconf automake make psmisc lsof net-tools vim python3

#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

#关闭selinux
setenforce 0 #临时关闭
#永久关闭
sed -i '/^SELINUX/ s/enforcing/disabled/' /etc/selinux/config

安装docker

卸载旧版本

yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine

安装yum-utils软件包（提供yum-config-manager 实用程序），配置加速源

yum install -y yum-utils
yum-config-manager \
--add-repo \
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

安装docker,设置开机自启

yum install -y docker-ce docker-ce-cli containerd.io #安装docker

systemctl start docker #启动docker
systemctl enable docker #设置docker自启动

配置 Docker使用systemd作为默认Cgroup驱动

每台服务器上都要操作，master和node上都要操作

cat <<EOF > /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

#重启docker
systemctl restart docker

关闭swap分区

每台服务器都需要操作

swapoff -a # 临时
vim /etc/fstab 永久  
#sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab #对这条语句进行注释 就是前面加个#号

重新命名主机，在所有主机上上添加如下命令，修改hosts文件

hostnamectl set-hostname master

hostnamectl set-hostname node1

hostnamectl set-hostname node2

修改主机名后使用su - root重新登陆

每台机器上的/etc/hosts文件都需要修改

su  - root

cat >> /etc/hosts << EOF 
192.168.10.133 master
192.168.10.132 node1
192.168.10.134 node2
192.168.10.135 node3
EOF

安装kubeadm,kubelet和kubectl

kubeadm --》k8s的管理程序--》在master上运行的--》建立整个k8s集群，背后是执行了大量的脚本，帮助我们去启动k8s

kubelet --》在node节点上用来管理容器的--》管理docker，告诉docker程序去启动容器 master和node通信用的--》管理docker，告诉docker程序去启动容器一个在集群中每个节点（node）上运行的代理。它保证容器（containers）都运行在 Pod 中。

kubectl --》在master上用来给node节点发号施令的程序，用来控制node节点的，告诉它们做什么事情的，是命令行操作的工具

集群里的每台服务器都需要安装

# 添加kubernetes YUM软件源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
#====
ubuntu里修改源，使用阿里云的源，参考文档
https://blog.csdn.net/wangyijieonline/article/details/105360138
#===ubuntu系统里
安装kubelet，kubeadm，kubectl
添加apt key以及源（所有主机）

#下载相关软件
sudo apt update && sudo apt install -y apt-transport-https curl

#下载apt-key.gpg
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

#添加k8s的阿里云的源
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >>/etc/apt/sources.list.d/kubernetes.list

sudo apt update
apt install -y kubelet kubeadm kubectl
systemctl enable --now kubelet
#===centos里执行
#安装kubeadm,kubelet和kubectl
yum install -y kubelet kubeadm kubectl
#设置开机自启
systemctl enable  kubelet

master主机执行

部署Kubernetes Master

# 部署Kubernetes Master
#提前准备coredns:1.8.4的镜像，后面需要使用,需要在每台机器上下载镜像
[root@master ~]# docker pull coredns/coredns:1.8.4
[root@master ~]# docker tag coredns/coredns:1.8.4 registry.aliyuncs.com/google_containers/coredns:v1.8.4

#初始化操作在master服务器上执行
[root@master ~]#kubeadm init \
--apiserver-advertise-address=192.168.10.133 \
--image-repository registry.aliyuncs.com/google_containers \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16
#192.168.0.17 是master的ip
# --service-cidr string Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")
# --pod-network-cidr string Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.

输出结果

   [addons] Applied essential addon: CoreDNS
   [addons] Applied essential addon: kube-proxy

   Your Kubernetes control-plane has initialized successfully!

   To start using your cluster, you need to run the following as a regular user:

   mkdir -p $HOME/.kube
   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown $(id -u):$(id -g) $HOME/.kube/config

   Alternatively, if you are the root user, you can run:

   export KUBECONFIG=/etc/kubernetes/admin.conf

   You should now deploy a pod network to the cluster.
   Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
   https://kubernetes.io/docs/concepts/cluster-administration/addons/

   Then you can join any number of worker nodes by running the following on each as root:


   kubeadm join 192.168.10.133:6443 --token 6plcy7.ulwz2zxqidc01h0k \
       --discovery-token-ca-cert-hash sha256:045c432d3a01cd63da84d2b0239b8fe805e0eec04c99f07f24667e9b849bdc91

   [root@master ~]#

按照提示操作

       mkdir -p $HOME/.kube
       sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
       sudo chown $(id -u):$(id -g) $HOME/.kube/config

## node节点执行

   建议：不要现在node节点上运行kubelet服务，不然会导致join失败，如果已经运行了，建议关闭，然后删除提示失败的文件和目录
   /etc/kubernetes/kubelet.conf
   /etc/kubernetes/pki/ca.crt
   rm -rf etc/kubernetes/{pki,kubelet.conf}
   ###########
   [root@node1 ~]# kubeadm join 192.168.0.17:6443 --token sxzm95.4blvhq1mnk3goyjo --discovery-token-ca-cert-hash sha256:419d644656e24a113f94d7f8e01a820bb4030e3c0a19a32542bc27730103189a

   [root@node1 ~]#
   第2台也加入集群
   [root@node3 ~]# kubeadm join 192.168.0.17:6443 --token sxzm95.4blvhq1mnk3goyjo --discovery-token-ca-cert-hash sha256:419d644656e24a113f94d7f8e01a820bb4030e3c0a19a32542bc27730103189a

#k8s里删除节点k8s-node1
       kubectl drain k8s-node1 --delete-emptydir-data --force --ignore-daemonsets node/k8s-node1
       kubectl delete node k8s-node1
输出结果

   [preflight] Running pre-flight checks
       [WARNING FileExisting-tc]: tc not found in system path
       [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.5. Latest validated version: 19.03
   [preflight] Reading configuration from the cluster...
   [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
   [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
   [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
   [kubelet-start] Starting the kubelet
   [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

   This node has joined the cluster:
   * Certificate signing request was sent to apiserver and a response was received.
   * The Kubelet was informed of the new secure connection details.

   Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

## 安装网络插件(在master节点执行)

kube-flannel.yml

   在master服务器上执行
   [root@master ~]# cat kube-flannel.yml
   ---
   apiVersion: policy/v1beta1
   kind: PodSecurityPolicy
   metadata:
   name: psp.flannel.unprivileged
   annotations:
   seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
   seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
   apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
   apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
   spec:
   privileged: false
   volumes:
   - configMap
   - secret
   - emptyDir
   - hostPath
   allowedHostPaths:
   - pathPrefix: "/etc/cni/net.d"
   - pathPrefix: "/etc/kube-flannel"
   - pathPrefix: "/run/flannel"
   readOnlyRootFilesystem: false
   # Users and groups
   runAsUser:
   rule: RunAsAny
   supplementalGroups:
   rule: RunAsAny
   fsGroup:
   rule: RunAsAny
   # Privilege Escalation
   allowPrivilegeEscalation: false
   defaultAllowPrivilegeEscalation: false
   # Capabilities
   allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
   defaultAddCapabilities: []
   requiredDropCapabilities: []
   # Host namespaces
   hostPID: false
   hostIPC: false
   hostNetwork: true
   hostPorts:
   - min: 0
   max: 65535
   # SELinux
   seLinux:
   # SELinux is unused in CaaSP
   rule: 'RunAsAny'
   ---
   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
   name: flannel
   rules:
   - apiGroups: ['extensions']
   resources: ['podsecuritypolicies']
   verbs: ['use']
   resourceNames: ['psp.flannel.unprivileged']
   - apiGroups:
   - ""
   resources:
   - pods
   verbs:
   - get
   - apiGroups:
   - ""
   resources:
   - nodes
   verbs:
   - list
   - watch
   - apiGroups:
   - ""
   resources:
   - nodes/status
   verbs:
   - patch
   ---
   kind: ClusterRoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
   name: flannel
   roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: flannel
   subjects:
   - kind: ServiceAccount
   name: flannel
   namespace: kube-system
   ---
   apiVersion: v1
   kind: ServiceAccount
   metadata:
   name: flannel
   namespace: kube-system
   ---
   kind: ConfigMap
   apiVersion: v1
   metadata:
   name: kube-flannel-cfg
   namespace: kube-system
   labels:
   tier: node
   app: flannel
   data:
   cni-conf.json: |
   {
   "name": "cbr0",
   "cniVersion": "0.3.1",
   "plugins": [
   {
   "type": "flannel",
   "delegate": {
   "hairpinMode": true,
   "isDefaultGateway": true
   }
   },
   {
   "type": "portmap",
   "capabilities": {
   "portMappings": true
   }
   }
   ]
   }
   net-conf.json: |
   {
   "Network": "10.244.0.0/16",
   "Backend": {
   "Type": "vxlan"
   }
   }
   ---
   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
   name: kube-flannel-ds
   namespace: kube-system
   labels:
   tier: node
   app: flannel
   spec:
   selector:
   matchLabels:
   app: flannel
   template:
   metadata:
   labels:
   tier: node
   app: flannel
   spec:
   affinity:
   nodeAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:
   nodeSelectorTerms:
   - matchExpressions:
   - key: kubernetes.io/os
   operator: In
   values:
   - linux
   hostNetwork: true
   priorityClassName: system-node-critical
   tolerations:
   - operator: Exists
   effect: NoSchedule
   serviceAccountName: flannel
   initContainers:
   - name: install-cni
   image: quay.io/coreos/flannel:v0.13.1-rc2
   command:
   - cp
   args:
   - -f
   - /etc/kube-flannel/cni-conf.json
   - /etc/cni/net.d/10-flannel.conflist
   volumeMounts:
   - name: cni
   mountPath: /etc/cni/net.d
   - name: flannel-cfg
   mountPath: /etc/kube-flannel/
   containers:
   - name: kube-flannel
   image: quay.io/coreos/flannel:v0.13.1-rc2
   command:
   - /opt/bin/flanneld
   args:
   - --ip-masq
   - --kube-subnet-mgr
   resources:
   requests:
   cpu: "100m"
   memory: "50Mi"
   limits:
   cpu: "100m"
   memory: "50Mi"
   securityContext:
   privileged: false
   capabilities:
   add: ["NET_ADMIN", "NET_RAW"]
   env:
   - name: POD_NAME
   valueFrom:
   fieldRef:
   fieldPath: metadata.name
   - name: POD_NAMESPACE
   valueFrom:
   fieldRef:
   fieldPath: metadata.namespace
   volumeMounts:
   - name: run
   mountPath: /run/flannel
   - name: flannel-cfg
   mountPath: /etc/kube-flannel/
   volumes:
   - name: run
   hostPath:
   path: /run/flannel
   - name: cni
   hostPath:
   path: /etc/cni/net.d
   - name: flannel-cfg
   configMap:
   name: kube-flannel-cfg

部署flannel

   [root@master ~]# kubectl apply -f kube-flannel.yml
   podsecuritypolicy.policy/psp.flannel.unprivileged created
   clusterrole.rbac.authorization.k8s.io/flannel created
   clusterrolebinding.rbac.authorization.k8s.io/flannel created
   serviceaccount/flannel created
   configmap/kube-flannel-cfg created
   daemonset.apps/kube-flannel-ds created
   [root@master ~]# ps aux|grep flannel
   root 320372 1.8 1.4 1265920 26824 ? Ssl 11:12 0:00 /opt/bin/flanneld --ip-masq --kube-subnet-mgr
   root 321071 0.0 0.0 12324 1088 pts/0 S+ 11:13 0:00 grep --color=auto flannel

查看集群状态

#等待节点准备完毕
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 161m v1.20.4
node1 Ready <none> 135m v1.20.4
node2 NotReady <none> 39s v1.20.4
过一段时间，再次查看node节点会是ready状态，大概2-3分钟时间
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 163m v1.20.4
node1 Ready <none> 136m v1.20.4
node2 Ready <none> 2m5s v1.20.4
[root@master ~]# kubectl get nodes -n kube-system -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master Ready control-plane,master 136m v1.22.1 192.168.0.17 <none> CentOS Linux 8 4.18.0-305.3.1.el8.x86_64 docker://20.10.8
node1 Ready <none> 131m v1.22.1 192.168.0.89 <none> CentOS Linux 7 (Core) 3.10.0-1062.el7.x86_64 docker://20.10.8
node2 Ready <none> 111s v1.22.1 192.168.0.11 <none> Ubuntu 20.04.3 LTS 5.4.0-80-generic docker://20.10.8
node3 Ready <none> 130m v1.22.1 192.168.0.19 <none> CentOS Linux 8 4.18.0-305.3.1.el8.x86_64 docker://20.10.8
[root@master ~]#

[root@master ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7f89b7bc75-fjq7n 1/1 Running 0 48m
coredns-7f89b7bc75-t5hnq 1/1 Running 0 48m
etcd-master 1/1 Running 0 48m
kube-apiserver-master 1/1 Running 0 48m
kube-controller-manager-master 1/1 Running 0 48m
kube-flannel-ds-kz8t9 1/1 Running 0 17m
kube-flannel-ds-r2q5s 1/1 Running 0 17m
kube-proxy-4jlr6 1/1 Running 0 48m
kube-proxy-ppsr6 1/1 Running 0 22m
kube-scheduler-master 1/1 Running 0 48m

测试

创建nginx的pod

启动一个pod，背后是运行nginx镜像
[root@new-k8s-master ~]# kubectl run sc-nginx --image=nginx --port=8080
pod/sc-nginx created
[root@new-k8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
sc-nginx 0/1 ContainerCreating 0 5s
[root@new-k8s-master ~]#
等一下，会启动pod，因为需要去下载镜像文件
[root@new-k8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
sc-nginx 1/1 Running 0 64s
[root@new-k8s-master ~]#
[root@new-k8s-master ~]# kubectl get pod -o wide 显示详细的内容
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
sc-nginx 1/1 Running 0 97s 10.244.1.2 node1 <none> <none>
[root@new-k8s-master ~]#

码农公寓