2、二进制安装K8s 之部署ETCD集群

2022-01-09 22:36:50

二进制安装K8s 之部署ETCD集群

一、下载安装cfssl，用于k8s证书签名

二进制包地址:https://pkg.cfssl.org/

所需软件包：

cfssl 1.6.0
cfssljson 1.6.0
cfssl-certinfo 1.6.0

wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssl_1.6.0_linux_amd64 -O cfssl

wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssljson_1.6.0_linux_amd64 -O cfssljson

wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssl-certinfo_1.6.0_linux_amd64 -O cfssl-certinfo

chmod +x cfssl*

mv cfssl* /usr/local/bin/

2、生成etcd证书

[x] 自签CA:

#生成默认的证书配置文件【可以省略此步骤】,如果没有证书配置文件模板可以使用

cfssl  print-defaults  config >ca-config.json

cfssl print-defaults csr >ca-csr.json

[x] 修改证书

cat > ca-config.json <<EOF

{

    "signing": {

        "default": {

            "expiry": "87600h"

        },

        "profiles": {

            "www": {

                "expiry": "87600h",

                "usages": [

                    "signing",

                    "key encipherment",

                    "server auth",

                    "client auth"

                ]

            }

        }

    }

}

EOF

cat > ca-csr.json <<EOF

{

    "CN": "etcd CA",

    "key": {

        "algo": "rsa",

        "size": 2048

    },

    "names": [

        {

            "C": "CN",

            "L": "Beijing",

            "ST": "Beijing"

        }

    ]

}

EOF

[x] 生成证书

生成ca.pem ca-key.pem 根证书文件

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

[x] 使用自签CA签发Etcd HTTPS证书

创建证书申请文件：

#注：上述文件hosts字段中IP为所有etcd节点的集群内部通信IP，一个都不能少！为了方便后期扩容可以多写几个预留的IP。

cat > server-csr.json <<EOF

{

    "CN": "etcd",

    "hosts": [

      "192.168.100.170",

      "192.168.100.171",

      "192.168.100.172",

      "192.168.100.173"

    ],

    "key": {

        "algo": "rsa",

        "size": 2048

    },

    "names": [

        {

            "C": "CN",

            "L": "BeiJing",

            "ST": "BeiJing"

        }

    ]

}

EOF

# 生成域名证书 server-key.pem server.pem

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

3、下载安装etcd

[x] 下载二进制包

#下载

wget https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz -O /data/download/

#解压

tar -zxvf etcd-v3.5.0-linux-amd64.tar.gz

#建议复制到/usr/local/bin/ 目录下

mv /data/download/etcd-v3.5.0-linux-amd64/{etcd,etcdctl} /usr/local/bin/

mv /data/download/etcd-v3.5.0-linux-amd64/{etcd,etcdctl} /data/k8s/etcd/bin/

[x] 创建etcd配置文件

cat > /data/etcd/config/etcd.conf << EOF

#[Member]

ETCD_NAME="etcd-1"

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_PEER_URLS="https://192.168.100.170:2380"

ETCD_LISTEN_CLIENT_URLS="https://192.168.100.170:2379"

#[Clustering]

ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.170:2380"

ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.170:2379"

ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.100.170:2380,etcd-2=https://192.168.100.171:2380,etcd-3=https://192.168.100.172:2380"

ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"

ETCD_INITIAL_CLUSTER_STATE="new"

EOF

[x] 3、systemd管理etcd

注意证书路径



cat > /usr/lib/systemd/system/etcd.service << EOF

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

[Service]

Type=notify

EnvironmentFile=/data/etcd/config/etcd.conf

ExecStart=/usr/local/bin/etcd \

--cert-file=/data/etcd/ssl/server.pem \

--key-file=/data/etcd/ssl/server-key.pem \

--peer-cert-file=/data/etcd/ssl/server.pem \

--peer-key-file=/data/etcd/ssl/server-key.pem \

--trusted-ca-file=/data/etcd/ssl/ca.pem \

--peer-trusted-ca-file=/data/etcd/ssl/ca.pem \

--logger=zap

Restart=on-failure

LimitNOFILE=65536

[Install]

WantedBy=multi-user.target

EOF

[x] 4、拷贝刚才生成的证书

把刚才生成的证书拷贝到配置文件中的路径：

cp /data/docker/TSL/etcd/*.pem /data/etcd/ssl/

[x] 5、启动并设置开机启动

systemctl daemon-reload

systemctl enable etcd

systemctl start etcd

6 、将上面节点1所有生成的文件拷贝到节点2和节点3

#复制整个目录

scp -r /data/etcd/* root@192.168.100.171:/data/etcd/

scp -r /data/etcd/* root@192.168.100.172:/data/etcd/

#复制systemd文件

scp /usr/lib/systemd/system/etcd.service root@192.168.100.171:/usr/lib/systemd/system/

scp /usr/lib/systemd/system/etcd.service root@192.168.100.172:/usr/lib/systemd/system/

#cp etcd 二进制文件 集群其他机器上操作

cp /data/etcd/bin/etc* /usr/local/bin/

#然后在节点2和节点3分别修改etcd.conf配置文件中的节点名称和当前服务器IP：

vi /opt/etcd/cfg/etcd.conf

#[Member]

ETCD_NAME="etcd-1"   # 修改此处，节点2改为etcd-2，节点3改为etcd-3

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_PEER_URLS="https://192.168.100.71:2380"   # 修改此处为当前服务器IP

ETCD_LISTEN_CLIENT_URLS="https://192.168.100.71:2379" # 修改此处为当前服务器IP

#[Clustering]

ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.71:2380" # 修改此处为当前服务器IP

ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.71:2379" # 修改此处为当前服务器IP

ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.100.71:2380,etcd-2=https://192.168.31.72:2380,etcd-3=https://192.168.31.73:2380"

ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"

ETCD_INITIAL_CLUSTER_STATE="new"

#启动，并设置开始启动

systemctl daemon-reload

systemctl enable etcd

systemctl start etcd

7、查看集群状态

systemctl status  etcd

● etcd.service - Etcd Server

   Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)

   Active: active (running) since 日 2021-07-18 19:04:49 CST; 18s ago

 Main PID: 1875 (etcd)

    Tasks: 8

   Memory: 33.5M

   CGroup: /system.slice/etcd.service

           └─1875 /usr/local/bin/etcd --cert-file=/data/k8s/etcd/ssl/server.pem --key-file=/data/k8s/etcd/ssl/server-key.pem --peer-cert-file=/data/k8s/etcd/ssl/server.pem --peer-key-file=/data/k8s/etcd/ssl/serve...

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.863+0800","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"1bd67ef396fd86"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.864+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-rea...bd67ef396fd86"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.865+0800","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-rea...bd67ef396fd86"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.866+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"7f0b6bf57639838f","to":"1bd67ef396fd8...eam MsgApp v2"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.866+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-wri...bd67ef396fd86"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.899+0800","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"7f0b6bf57639838f","to":"1bd67ef396fd8...tream Message"}

7月 18 19:04:51 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:51.899+0800","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-wri...bd67ef396fd86"}

7月 18 19:04:53 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:53.988+0800","caller":"etcdserver/server.go:2481","msg":"updating cluster version using v2 API","from":"3.0","to":"3.5"}

7月 18 19:04:53 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:53.991+0800","caller":"membership/cluster.go:523","msg":"updated cluster version","cluster-id":"a89a4473c024c0a2","local....0","to":"3.5"}

7月 18 19:04:53 master01 etcd[1875]: {"level":"info","ts":"2021-07-18T19:04:53.991+0800","caller":"etcdserver/server.go:2500","msg":"cluster version is updated","cluster-version":"3.5"}

Hint: Some lines were ellipsized, use -l to show in full.



etcdctl --cacert=/data/etcd/ssl/ca.pem \

--cert=/data/etcd/ssl/server.pem \

--key=/data/etcd/ssl/server-key.pem \

--endpoints="https://192.168.0.170:2379,https://192.168.100.171:2379,https://192.168.100.172:2379"  endpoint status -w table

etcdctl --cacert=/data/etcd/ssl/ca.pem \

--cert=/data/etcd/ssl/server.pem \

--key=/data/etcd/ssl/server-key.pem \

--endpoints="https://192.168.100.170:2379,https://192.168.100.171:2379,https://192.168.100.172:2379"  endpoint health

+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

|          ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |

+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

| https://192.168.100.170:2379 |  32e07c4d987eefc |   3.5.0 |   29 kB |      true |      false |         2 |          9 |                  9 |        |

| https://192.168.100.171:2379 | 7ec2542a2723e9e3 |   3.5.0 |   20 kB |     false |      false |         2 |          9 |                  9 |        |

| https://192.168.100.172:2379 | 2186647c238c4402 |   3.5.0 |   20 kB |     false |      false |         2 |          9 |                  9 |        |

+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

https://192.168.100.170:2379 is healthy: successfully committed proposal: took = 32.498535ms

https://192.168.100.171:2379 is healthy: successfully committed proposal: took = 37.070854ms

https://192.168.100.172:2379 is healthy: successfully committed proposal: took = 37.475938ms

#如果输出上面信息，就说明集群部署成功。如果有问题第一步先看日志：/var/log/message 或 journalctl -u etcd

码农公寓

二进制安装K8s 之 部署ETCD集群

一、下载安装cfssl，用于k8s证书签名

2、生成etcd证书

3、下载安装etcd

相关文章

二进制安装K8s 之部署ETCD集群