文章目录
master高可用部署流程
- 去这篇博客，内容过多，需要6台虚拟机，有条件的建议跟着实验完整做一遍，一遍更深层次的理解高可用：
【kubernetes】k8s集群高可用部署安装和概念详细说明【含离线部署】，客户端连接haproxy访问高可用流程
客户端连接haproxy访问高可用集群
环境确认与准备【必看】
- 客户端连接haproxy访问高可用集群，这句话直观吗？
是这个意思，就是我们使用任意集群外的主机，使用kubeconfig的形式连接到haproxy，然后haproxy会自动转发到master，所以只需要连接haproxy，就可以实现集群高可用了【2个master死其中一个无所谓的】 - 首先确保上面文章中高可用部署完整且测试正常，然后6台虚拟机都需要开机
[root@master1-163 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master1-163 Ready control-plane,master 22h v1.21.1
master2-162 Ready control-plane,master 21h v1.21.1
worker-165 Ready <none> 17h v1.21.1
[root@master1-163 ~]#
[root@master1-163 ~]# kubectl get pods -A -owide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system calico-kube-controllers-78d6f96c7b-nwbmt 1/1 Running 0 16h 10.244.139.67 worker-165 <none> <none>
kube-system calico-node-nh977 1/1 Running 0 16h 192.168.59.162 master2-162 <none> <none>
kube-system calico-node-s89tx 1/1 Running 1 16h 192.168.59.163 master1-163 <none> <none>
kube-system calico-node-vt5dn 1/1 Running 0 16h 192.168.59.165 worker-165 <none> <none>
kube-system coredns-545d6fc579-6l9xs 1/1 Running 0 22h 10.244.139.66 worker-165 <none> <none>
kube-system coredns-545d6fc579-mrm2w 1/1 Running 0 22h 10.244.139.65 worker-165 <none> <none>
kube-system kube-apiserver-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-apiserver-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-controller-manager-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-controller-manager-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-proxy-kp8p6 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-proxy-kqg72 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-proxy-nftgv 1/1 Running 0 17h 192.168.59.165 worker-165 <none> <none>
kube-system kube-scheduler-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-scheduler-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
[root@master1-163 ~]#
- 最后，找一台客户端用来访问haproxy，找一台不属于集群一部分的机器。
我这就用之前的etcd1来做客户端主机吧
Last login: Fri Nov 26 12:58:16 2021 from 192.168.59.1
[root@etcd1 ~]#
[root@etcd1 ~]# ip a | grep 59
inet 192.168.59.156/24 brd 192.168.59.255 scope global ens32
[root@etcd1 ~]#
客户端连接happroxy说明
-
client连接happroxy有2种方式
- 1、kubeconfig的方式
- 2、token的方式
-
我之前文章中对kubeconfig和token都有说明，想了解的看这篇
【Kubernetes】k8s的安全管理详细说明【k8s框架说明、token验证和kubeconfig验证详细说明】 -
我这使用kubeconfig的方式来访问
kubeconfig配置【master上操作】
-
我也是跟着上面文章中配置的，我这不对命令做说明，不清楚的自行去看上面那篇文章中的说明哈
-
在任意一个master上操作即可，我在master1上操作吧
下面连接地址改为happroxy的地址，已经回车一个空行，并做注释了，注意看
[root@master1-163 ~]# mkdir sefe
[root@master1-163 ~]# cd sefe
[root@master1-163 sefe]# openssl genrsa -out ccx.key 2048
Generating RSA private key, 2048 bit long modulus
..........................+++
....+++
e is 65537 (0x10001)
[root@master1-163 sefe]# openssl req -new -key ccx.key -out ccx.csr -subj "/CN=ccx/O=cka2021"
[root@master1-163 sefe]# cat ccx.csr | base64 | tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==[root@master1-163 sefe]#
[root@master1-163 sefe]# cat csr.yaml
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: ccx
spec:
groups:
- system:authenticated
request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
usages:
- client auth
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl apply -f csr.yaml
Warning: certificates.k8s.io/v1beta1 CertificateSigningRequest is deprecated in v1.19+, unavailable in v1.22+; use certificates.k8s.io/v1 CertificateSigningRequest
certificatesigningrequest.certificates.k8s.io/ccx created
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
ccx 23s kubernetes.io/legacy-unknown kubernetes-admin Pending
[root@master1-163 sefe]#
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl certificate approve ccx
certificatesigningrequest.certificates.k8s.io/ccx approved
[root@master1-163 sefe]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
ccx 2m32s kubernetes.io/legacy-unknown kubernetes-admin Approved,Issued
[root@master1-163 sefe]# kubectl get csr/ccx -o jsonpath='{.status.certificate}' | base64 -d > ccx.crt
[root@master1-163 sefe]#
[root@master1-163 sefe]# cat ccx.crt
-----BEGIN CERTIFICATE-----
MIIDBjCCAe6gAwIBAgIRAJntISUbREJeqXDV7z2+HQUwDQYJKoZIhvcNAQELBQAw
FTETMBEGA1UEAxMKa3ViZXJuZXRlczAeFw0yMTExMzAwMjUwMjdaFw0yMjExMzAw
MjUwMjdaMCAxEDAOBgNVBAoTB2NrYTIwMjExDDAKBgNVBAMTA2NjeDCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBANuC5fTmLcwzoW02OyfYtsVYyKgL32iq
WfEaoTZyZYA0v2CEk16TsujjSS+skH+4wtbzjYJJRRX7ys5Neu7UL/q13flFyRcJ
HT2Spn1ZcsTGQ1gV0ByKE0qGfEqpv5JwD+AeQ4mGNBrdSJ79hM2S5B+NSuY9gdKc
cwAuR0K0tLGPRR9XK474RvJP/KqvuE0ab/jAMZyrTUF16/Hvzucyrb0Z8NOIWi9B
M04hvPD/W9zyUTOlp/DJlJK1RDbvh0STRdvFJg4kz9Q8n7R702Lc2UY9NvRPike2
BHfjZ0p5zbc10hxNFtQJrdybWIckq96G7hc+mB8P36SvSK7In5YZI5MCAwEAAaNG
MEQwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAW
gBRFBBI4o0uRg0+Dv6LzwLmyou1PCDANBgkqhkiG9w0BAQsFAAOCAQEAJNE1NuAs
o5cnRU+65ys/+xxBt3Fg8DMj0HokLumvFtZ7CDahX5fPHt6YynAfulgdbhba01UV
c854sOfDO8xYEuaRLCKn+a+yYH7QCOztlJGHlaKiYk7JERdj0u199gpJ+ANLoSQP
fJVwwfclioFf16UDPXNocSJQrjWih34HNrudCy1XjPOu7etjT2ICQ1LD04w77Ls/
speOYBrOQFR33Utn0s/xoGI8ExPzCSpT1Zc/JwRDUoaD6Lu83XaMD24ip+Jj13TY
uI62+u0VfMvp8eiS6MigwP7w7vMD6XqyDF1yXmsnAFspEhuiJcG6fGO2OnkcRb8P
Q5c+ijY0QDP3GA==
-----END CERTIFICATE-----
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl create clusterrolebinding test1 --clusterrole=cluster-admin --user=ccx
clusterrolebinding.rbac.authorization.k8s.io/test1 created
[root@master1-163 sefe]# cp /etc/kubernetes/pki/ca.crt .
[root@master1-163 sefe]#
# 下面地址改为：happroxy的ip
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-cluster cluster1 --server=https://192.168.59.164:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "cluster1" set.
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-credentials ccx --client-certificate=ccx.crt --client-key=ccx.key --embed-certs=true
User "ccx" set.
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-context context1 --cluster=cluster1 --namespace=default --user=ccx
Context "context1" created.
[root@master1-163 sefe]# vi kc1
[root@master1-163 sefe]# cat kc1
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.59.164:6443
name: cluster1
contexts:
- context:
cluster: cluster1
namespace: default
user: ccx
name: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccx
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@master1-163 sefe]#
- 至此呢，kubeconfig文件就准备好了，我们重命名为kc2，并拷贝到客户端主机上吧【之前有个kc1，免得冲突】
[root@master1-163 sefe]# mv kc1 kc2
[root@master1-163 sefe]# scp kc2 192.168.59.156:~
The authenticity of host '192.168.59.156 (192.168.59.156)' can't be established.
ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.
ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.59.156' (ECDSA) to the list of known hosts.
root@192.168.59.156's password:
kc2 100% 5507 4.0MB/s 00:00
[root@master1-163 sefe]#
客户端测试
说明
- 上面我们已经把kc2文件拷贝到当前服务器上了
并且kc2中连接ip是happroxy的ip地址，你确认下是否有误。
所有配置文件内容如下
[root@etcd1 ~]# cat kc2
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.59.164:6443
name: cluster1
contexts:
- context:
cluster: cluster1
namespace: default
user: ccx
name: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccx
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@etcd1 ~]#
- 为什么要连接到happroxy，因为我们在happroxy上配置了，只要访问当前ip的6443端口，就会自动转发到master上
我们在happroxy上查看规则
[root@haproxy-164 haproxy]# netstat -ntlp | grep 6443
tcp 0 0 0.0.0.0:6443 0.0.0.0:* LISTEN 2504/haproxy
[root@haproxy-164 haproxy]# tail /etc/haproxy/haproxy.cfg
server app1 127.0.0.1:5001 check
server app2 127.0.0.1:5002 check
server app3 127.0.0.1:5003 check
server app4 127.0.0.1:5004 check
listen k8s-lb *:6443
mode tcp
balance roundrobin
server s1 192.168.59.163:6443 weight 1
server s2 192.168.59.162:6443 weight 1
[root@haproxy-164 haproxy]#
测试
- 如下，没有kubectl命令的话，需要先安装一个命令哈
[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get nodes
-bash: kubectl: 未找到命令
[root@etcd1 ~]#
# 我是离线安装的
[root@etcd1 kubelet]# rpm -ivhU * --nodeps --force
警告：3944a45bec4c99d3489993e3642b63972b62ed0a4ccb04cc7655ce0467fddfef-kubectl-1.21.1-0.x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID 3e1ba8d5: NOKEY
准备中... ################################# [100%]
正在升级/安装...
1:socat-1.7.3.2-2.el7 ################################# [ 9%]
2:libnetfilter_queue-1.0.2-2.el7_2 ################################# [ 18%]
3:libnetfilter_cttimeout-1.0.0-7.el################################# [ 27%]
4:libnetfilter_cthelper-1.0.0-11.el################################# [ 36%]
5:libnetfilter_conntrack-1.0.6-1.el################################# [ 45%]
6:conntrack-tools-1.4.4-7.el7 ################################# [ 55%]
7:kubernetes-cni-0.8.7-0 ################################# [ 64%]
8:kubelet-1.21.1-0 ################################# [ 73%]
9:cri-tools-1.19.0-0 ################################# [ 82%]
10:kubectl-1.21.1-0 ################################# [ 91%]
11:kubeadm-1.21.1-0 ################################# [100%]
[root@etcd1 kubelet]# systemctl enable kubelet.service --now
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@etcd1 kubelet]#
- 再来，测试
没问题，完美啊，卧槽~ 一次性成功了 这次没有处理报错 好感动啊
[root@etcd1 kubelet]# cd
[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master1-163 Ready control-plane,master 23h v1.21.1
master2-162 Ready control-plane,master 22h v1.21.1
worker-165 Ready <none> 18h v1.21.1
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME STATUS AGE
default Active 23h
kube-node-lease Active 23h
kube-public Active 23h
kube-system Active 23h
ns1 Active 100m
ns2 Active 99m
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl create ns ns3
namespace/ns3 created
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME STATUS AGE
default Active 23h
kube-node-lease Active 23h
kube-public Active 23h
kube-system Active 23h
ns1 Active 100m
ns2 Active 99m
ns3 Active 6s
[root@etcd1 ~]#
更好的部署master高可用的方式
-
这是国内的一个方式，叫做
sealos收费的，但是一年就几十块钱，很便宜，而且一件部署，炒鸡方便啊~ 上面手动部署的太麻烦了，如果不想折腾的，那么sealos绝对是你的首选哦 -
官网已经做了很详细的使用流程了，而且很简单
我是干技术的，所以呢我还是喜欢使用上面自己动手的配置方法，我不会使用这个，所以我这就不多累赘了，感兴趣的小伙伴去官网注册试试把 ~