（二）kubernetes集群搭建（二进制方式）之--搭建etcd

2024-01-20 18:35:34

kubernetes集群搭建（二进制方式）之--搭建etcd

Etcd是一个分布式键值存储系统，也是k8s使用的数据存储系统，所以先搭建etcd，为解决单点故障，采用集群方式部署，两台机器都部署etcd。

1.准备证书生成工具cfssl

cfssl是一个开源的证书管理工具，使用json文件生成证书，在master上操作生成证书，etcd的通信需要认证证书通信

#下载工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
#添加执行权限
chmod +x cfssl*
#放入可执行目录
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

2.生成Etcd证书

（1）自签证书办法机构

创建工作目录

mkdir -p ~/TLS/{etcd,k8s}
cd TLS/etcd

自签CA：

cat > ca-config.json<< EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
        "expiry": "87600h",
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ]
      }
    }
  }
}
EOF

cat > ca-csr.json<< EOF
{
  "CN": "etcd CA",
  "key": {
  "algo": "rsa",
  "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "Beijing",
      "ST": "Beijing"
    }
  ]
}
EOF

生成证书：

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

（2）使用自签CA签发Etcd HTTPS 证书

创建证书申请文件：

cat > server-csr.json<< EOF
{
    "CN": "etcd",
    "hosts": [
        "172.21.32.17",
        "172.21.32.15",
        "172.21.32.16",
        "172.21.32.18",
        "172.21.32.19"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [{
        "C": "CN",
        "L": "BeiJing",
        "ST": "BeiJing"
    }]
}
EOF

注：上述文件hosts字段中ip为所有etcd节点的集群内部通信ip，一个不能少，为了方便以后扩容，可以多写几个预留的ip
生成证书：

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

3.下载二进制文件

下载地址：https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz
以下在master节点操作，操作完事将生成文件拷贝到node节点
获取二进制文件：

#创建源文件目录用于存储
mkdir -p /data/src
cd /data/src
yum -y install wget
wget https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz

4.部署etcd集群

（1）创建工作目录并解压二进制包

mkdir /data/etcd/{bin,cfg,ssl} -p
tar fx etcd-v3.4.9-linux-amd64.tar.gz -C /data/etcd/
cd /data/etcd/etcd-v3.4.9-linux-amd64
mv ./{etcd,etcdctl} /data/etcd/bin/

（2）创建etcd配置文件

cat > /data/etcd/cfg/etcd.conf <<EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://172.21.32.17:2380"
ETCD_LISTEN_CLIENT_URLS="https://172.21.32.17:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://172.21.32.17:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://172.21.32.17:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://172.21.32.17:2380,etcd-2=https://172.21.32.15:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

ETCD_NAME：节点名称，集群中唯一
ETCD_DATA_DIR：数据目录
ETCD_LISTEN_PEER_URLS：集群通信监听地址
ETCD_LISTEN_CLIENT_URLS：客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEER_URLS：集群通告地址
ETCD_ADVERTISE_CLIENT_URLS：客户端通告地址
ETCD_INITIAL_CLUSTER：集群节点地址
ETCD_INITIAL_CLUSTER_TOKEN：集群Token
ETCD_INITIAL_CLUSTER_STATE：加入集群的当前状态，new 是新集群，existing 表示加入已有集群

（3）systemd管理etcd

cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/data/etcd/cfg/etcd.conf
ExecStart=/data/etcd/bin/etcd \
--cert-file=/data/etcd/ssl/server.pem \
--key-file=/data/etcd/ssl/server-key.pem \
--peer-cert-file=/data/etcd/ssl/server.pem \
--peer-key-file=/data/etcd/ssl/server-key.pem \
--trusted-ca-file=/data/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/data/etcd/ssl/ca.pem \
--logger=zap
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF

（4）拷贝刚才生成的证书

cp ~/TLS/etcd/ca*pem ~/TLS/etcd/server*pem /data/etcd/ssl/

（5）将刚刚master上的所有生成的文件拷贝到node节点

scp -r /data root@172.21.32.15:/
scp /usr/lib/systemd/system/etcd.service root@172.21.32.15:/usr/lib/systemd/system/

（6）在node节点更改etcd的配置文件

#[Member]
ETCD_NAME="etcd-2"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://172.21.32.15:2380"
ETCD_LISTEN_CLIENT_URLS="https://172.21.32.15:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://172.21.32.15:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://172.21.32.15:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://172.21.32.17:2380,etcd-2=https://172.21.32.15:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

（7）master和node同时操作

#加载启动文件
systemctl daemon-reload
#启动etcd
systemctl start etcd
#开机自启动
systemctl enable etcd
#查看进程状态
systemctl status etcd

（8）验证

/data/etcd/bin/etcdctl --cacert=/data/etcd/ssl/ca.pem --cert=/data/etcd/ssl/server.pem --key=/data/etcd/ssl/server-key.pem --endpoints="https://172.21.32.17:2379,https://172.21.32.15:2379" endpoint health

看到如下说明etcd集群搭建完成
https://172.21.32.17:2379 is healthy: successfully committed proposal: took = 9.011409ms
https://172.21.32.15:2379 is healthy: successfully committed proposal: took = 9.168965ms

码农公寓