Kubernetes多运营商云服务器部署(kubeadm+ipvs+flannel)

2023-12-09 21:04:20

服务器：轻量级阿里云+轻量级腾讯云

系统要求：OS7

安装Docker

安装命令

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum install wget

yum makecache

wget https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm

yum -y install containerd.io-1.2.6-3.3.el7.x86_64.rpm

yum install docker-ce docker-ce-cli containerd.io

启动：systemctl enable docker --now

如果已经有docker注意版本，低版本需要修改(具体需不需要可以先改了，如果docker重启启动不起来，就说明不需要，删掉即可)

修改docker驱动：

vim /usr/lib/systemd/system/docker.service

在ExecStart命令结尾添加 --exec-opt native.cgroupdriver=systemd

安装K8S

以下一些命令参考自网上的一些安装教程

cat > k8s.conf <<EOF
#开启网桥模式
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
#开启转发
net.ipv4.ip_forward = 1
##关闭ipv6
net.ipv6.conf.all.disable_ipv6=1
EOF
cp k8s.conf /etc/sysctl.d/k8s.conf
sysctl -p /etc/sysctl.d/k8s.conf

setenforce 0

# 设置系统时区为 中国/上海
timedatectl set-timezone Asia/Shanghai
# 将当前的UTC时间写入硬件时钟
timedatectl set-local-rtc 0
# 重启依赖于系统时间的服务
systemctl restart rsyslog
systemctl restart crond

systemctl stop postfix && systemctl disable postfix

mkdir /var/log/journal # 持久化保存日志的目录
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
[Journal]
# 持久化
Storage=persistent
# 压缩历史日志
Compress=yes
SysnIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
# 最大占用空间 10G
SystemMaxUse=10G
# 单日志文件最大 200M
SystemMaxFileSize=200M
# 日志保存时间 2 周
MaxRetentionSec=2week
# 不将日志转发到 syslog
ForwardToSyslog=no
EOF
systemctl restart systemd-journald

modprobe br_netfilter

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4


swapoff -a

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装三大件

yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9 --disableexcludes=kubernetes

systemctl enable --now kubelet

开放端口（来自网络）

10250/10260 TCP端口：给kube-schedule、kube-controll，kube-proxy、kubelet等使用

6443 TCP端口：给kube-apiserver使用

2379 2380 2381 TCP商品：ETCD使用

8472 UDP端口：vxlan使用端口

设置虚拟网卡

cat > /etc/sysconfig/network-scripts/ifcfg-eth0:1 <<EOF
BOOTPROTO=static
DEVICE=eth0:1
IPADDR=公网IP
PREFIX=32
TYPE=Ethernet
USERCTL=no
ONBOOT=yes
EOF

systemctl restart network

# 查看
ip addr

以上master和node都需要执行

以下master执行↓

配置kubeadm

kubeadm-config.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kubernetesVersion: v1.20.13
apiServer:
  certSANs:
  - aliyun-master
  - 公网
  - 内网
  - 10.96.0.1
controlPlaneEndpoint: 公网:6443
networking:
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
--- 
apiVersion: kubeproxy-config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
featureGates:
  SupportIPVSProxyMode: true
mode: ipvs

安装

kubeadm init --config=kubeadm-config.yaml

初始化成功后master执行生成的命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubeadm join xx.xx.xx.xx:6443 --token dfijdth \
 --discovery-token-ca-cert-hash dfsdfsdfsdfg9a460f44b118050091245c1d

安装ipvs

yum install ipvsadm

修改apiserver配置文件

vim /etc/kubernetes/manifests/kube-apiserver.yaml

spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=公网IP
    - --bind-address=0.0.0.0

安装网络插件flannel

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

修改配置文件对比修改缺什么补充什么

containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.15.1
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        - --public-ip=$(PUBLIC_IP)
        - --iface=eth0
 
 # env下
 env:
 - name: PUBLIC_IP
   valueFrom:          
     fieldRef:          
       fieldPath: status.podIP

然后再加入node节点就可以了

在此分享几个学习的网站

Kubernetes教程 | Kuboard

最后验证效果

查看ipvs