基于Docker部署ETCD集群
关于ETCD要不要使用TLS?
首先TLS的目的是为了鉴权为了防止别人任意的连接上你的etcd集群。其实意思就是说如果你要放到公网上的ETCD集群,并开放端口,我建议你一定要用TLS。
如果你的ETCD集群跑在一个内网环境比如(VPC环境),而且你也不会开放ETCD端口,你的ETCD跑在防火墙之后,一个安全的局域网中,那么你用不用TLS,都行。
优化参数
- --auto-compaction-retention
- 由于ETCD数据存储多版本数据,随着写入的主键增加历史版本需要定时清理,默认的历史数据是不会清理的,数据达到2G就不能写入,必须要清理压缩历史数据才能继续写入;所以根据业务需求,在上生产环境之前就提前确定,历史数据多长时间压缩一次;推荐一小时压缩一次数据这样可以极大的保证集群稳定,减少内存和磁盘占用
- --max-request-bytes
- etcd Raft消息最大字节数,ETCD默认该值为1.5M; 但是很多业务场景发现同步数据的时候1.5M完全没法满足要求,所以提前确定初始值很重要;由于1.5M导致我们线上的业务无法写入元数据的问题,我们紧急升级之后把该值修改为默认32M,但是官方推荐的是10M,大家可以根据业务情况自己调整
- --quota-backend-bytes
- ETCD db数据大小,默认是2G,当数据达到2G的时候就不允许写入,必须对历史数据进行压缩才能继续写入;参加1里面说的,我们启动的时候就应该提前确定大小,官方推荐是8G,这里我们也使用8G的配置
Docker安装ETCD
请依次在你规划好的etcd机器上运行即可
etcd-s1
mkdir -p /var/etcd
docker rm etcd1 -f
rm -rf /var/etcd
docker run --restart=always --net host -it --name etcd1 -d \
-v /var/etcd:/var/etcd \
-v /etc/localtime:/etc/localtime \
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.2. \
etcd --name etcd-s1 \
--auto-compaction-retention= --max-request-bytes= --quota-backend-bytes= \
--data-dir=/var/etcd/etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://192.168.150.141:2380 \
--advertise-client-urls http://192.168.150.141:2379,http://192.168.150.141:2380 \
-initial-cluster-token etcd-cluster \
-initial-cluster "etcd-s1=http://192.168.150.141:2380,etcd-s2=http://192.168.150.142:2380,etcd-s3=http://192.168.150.143:2380" \
-initial-cluster-state new
etcd-s2
mkdir -p /var/etcd
docker rm etcd2 -f
rm -rf /var/etcd
docker run --restart=always --net host -it --name etcd2 -d \
-v /var/etcd:/var/etcd \
-v /etc/localtime:/etc/localtime \
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.2. \
etcd --name etcd-s2 \
--auto-compaction-retention= --max-request-bytes= --quota-backend-bytes= \
--data-dir=/var/etcd/etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://192.168.150.142:2380 \
--advertise-client-urls http://192.168.150.142:2379,http://192.168.150.142:2380 \
-initial-cluster-token etcd-cluster \
-initial-cluster "etcd-s1=http://192.168.150.141:2380,etcd-s2=http://192.168.150.142:2380,etcd-s3=http://192.168.150.143:2380" \
-initial-cluster-state new
etcd-s3
mkdir -p /var/etcd
docker rm etcd3 -f
rm -rf /var/etcd
docker run --restart=always --net host -it --name etcd3 -d \
-v /var/etcd:/var/etcd \
-v /etc/localtime:/etc/localtime \
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.2. \
etcd --name etcd-s3 \
--auto-compaction-retention= --max-request-bytes= --quota-backend-bytes= \
--data-dir=/var/etcd/etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://192.168.150.143:2380 \
--advertise-client-urls http://192.168.150.143:2379,http://192.168.150.143:2380 \
-initial-cluster-token etcd-cluster \
-initial-cluster "etcd-s1=http://192.168.150.141:2380,etcd-s2=http://192.168.150.142:2380,etcd-s3=http://192.168.150.143:2380" \
-initial-cluster-state new
验证
➜ ETCDCTL_API= etcdctl member list
410feb26f4fa3c7f: name=etcd-s1 peerURLs=http://192.168.150.141:2380 clientURLs=http://192.168.150.141:2379,http://192.168.150.141:2380
56fa117fc503543c: name=etcd-s3 peerURLs=http://192.168.150.143:2380 clientURLs=http://192.168.150.143:2379,http://192.168.150.143:2380
bc4d900274366497: name=etcd-s2 peerURLs=http://192.168.150.142:2380 clientURLs=http://192.168.150.142:2379,http://192.168.150.142:2380 ➜ ETCDCTL_API= etcdctl cluster-health
member 410feb26f4fa3c7f is healthy: got healthy result from http://192.168.150.141:2379
member 56fa117fc503543c is healthy: got healthy result from http://192.168.150.143:2379
member bc4d900274366497 is healthy: got healthy result from http://192.168.150.142:2379
cluster is healthy
到此ETCD集群部署完毕。