二进制方式搭建Kubernetes 1.19.3高可用集群(五)——部署dashboard
本文将介绍在二进制部署的k8s集群中部署dashboar 2.0.4，并解决部署过程中metrics-server无法启动的问题
部署dashboard
首先，根据官方文档来，下载配置文件(官方文档地址：https://github.com/kubernetes/dashboard)
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml -O dashboard-deploy.yaml由于镜像在国外，下载可能会比较慢，所有我把镜像放到了阿里云上，可以按需替换下镜像
kubernetesui/dashboard:v2.0.4 替换为 registry.cn-shanghai.aliyuncs.com/jieee/dashboard:v2.0.4
kubernetesui/metrics-scraper:v1.0.4 替换为 registry.cn-shanghai.aliyuncs.com/jieee/metrics-scraper:v1.0.4
然后直接部署
kubectl apply -f dashboard-deploy.yaml
# 检查pod和service(默认的namespace是kubernetes-dashboard)
kubectl get pod -n kubernetes-dashboard
#NAME READY STATUS RESTARTS AGE
#dashboard-metrics-scraper-7b59f7d4df-bj66m 1/1 Running 0 2m
#kubernetes-dashboard-7df8bc567d-slbhs 1/1 Running 0 2m
kubectl get svc -n kubernetes-dashboard
#NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
#dashboard-metrics-scraper ClusterIP 10.120.5.204 <none> 8000/TCP 2m
#kubernetes-dashboard ClusterIP 10.120.209.68 <none> 443/TCP 2m至此，dashboard就部署完成了，然后我们就可以在浏览器中输入地址访问了（我这里的地址是https://10.120.209.68）
注意：由于dashboard使用了自签证书，所有chrome浏览器可能无法访问，使用Firefox可以正常访问
生成TOKEN
打开网页后，需要我们登陆
支持dashboard支持2种方式登陆，一般我们选择使用Token方式，先来创建一个Service Account
dashboard-rbac.yaml(这里我直接赋予了cluster-admin角色)
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: dashboard-admin
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin-bind-cluster-role
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin
namespace: kubernetes-dashboard部署并生成token
kubectl apply -f dashboard-rbac.yaml
#获取TOKEN
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep dashboard-admin | awk '{print $1}')
#Name: dashboard-admin-token-grxgp
#Namespace: kubernetes-dashboard
#Labels: <none>
#Annotations: kubernetes.io/service-account.name: dashboard-admin
# kubernetes.io/service-account.uid: 440d60e7-f75b-429f-ad2b-1a56d33e47c8
#
#Type: kubernetes.io/service-account-token
#
#Data
#====
#ca.crt: 1363 bytes
#namespace: 20 bytes
#token: eyJhbGciOiJSUzI1NiIsImtpZCI6InZmWF9vS29UWE53bVhKbkdUY3ZpLXdqYlBHc3VCUzdiamMzLS1FMDZhQUEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tZ3J4Z3AiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNDQwZDYwZTctZjc1Yi00MjlmLWFkMmItMWE1NmQzM2U0N2M4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.Uerz4ERXLeyKDfuNW_l-K_3xr3lh4Iyc8B5U_TnW8tlWgrYAcijTF86QESprolDmhn7s7RqVwrfUAHvmKoI_d08ApTWouu1lnoGIsn-qUovYOtAnpr-sal4TTWu9tjScodqklOw1WrICUiUFxcEN1939ERqx2oESYiKUuT2yEt2stMGUmp02QkmyiYtfk5a6sZ14LcyLL_mtC09hF4vW4dz2_QdP3qVd6l-RHS5NDFnB4bBz8m6TG6h2kY09tiGcFgjNfkQhFdy6L0F_jczufj39MrcRWofxROGKNo_vq2sSidekODjpp6TAIF43k51gW9T_qhUnrflemJAbUseqnw最后得到的一长串token就是登陆所需的token
登陆后就能看到整个集群的状态了
可是我们发现列表中，CPU和内存使用率都是空的，这是因为我们还没有安装metrics-server
安装metrics-server
官方文档：https://github.com/kubernetes-sigs/metrics-server
按照文档，我们先下载配置文件
wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml -O metrics-server.yaml同样的，我们替换一下镜像地址
k8s.gcr.io/metrics-server/metrics-server:v0.3.7 替换为 registry.cn-shanghai.aliyuncs.com/jieee/metrics-server:v0.3.7
然后部署
kubectl apply -f metrics-server.yaml
#检查pod状态
kubectl get pod -n kube-system | grep metrics-server
# metrics-server-f964c4474-t5sx9 1/1 Running 0 2m可以看到pod已经正常运行了。
然而，当我们回到dashboard中，发现CPU和内存信息还是没有出来，我们先来看一下pod日志
kubectl logs metrics-server-f964c4474-t5sx9 -n kube-system
#...
#E1107 05:15:45.224261 1 configmap_cafile_content.go:243] kube-system/extension-apiserver-authentication failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
#E1107 05:15:45.225200 1 configmap_cafile_content.go:243] key failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"我们发现了这两条错误日志，原来是我们部署apiserver是没有开启聚合功能，那我们就来开启一下吧
创建证书
cat > proxy-client-csr.json<<EOF
{
"CN": "aggregator",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Hangzhou",
"L": "Hangzhou",
"O": "system:masters",
"OU": "System"
}
]
}
EOF
#创建证书
cfssl gencert -ca=../ca.pem -ca-key=../ca-key.pem -config=../ca-config.json -profile=kubernetes proxy-client-csr.json | cfssljson -bare proxy-client
#分发证书至所有master节点
scp proxy-client*.pem root@10.0.50.101:/etc/kubernetes/pki/
scp proxy-client*.pem root@10.0.50.102:/etc/kubernetes/pki/
scp proxy-client*.pem root@10.0.50.103:/etc/kubernetes/pki/
修改apiserver的service文件
在启动命令中添加以下参数
vi /etc/systemd/system/kube-apiserver.service
...
--proxy-client-cert-file=/etc/kubernetes/pki/proxy-client.pem \
--proxy-client-key-file=/etc/kubernetes/pki/proxy-client-key.pem \
--runtime-config=api/all=true \
--requestheader-client-ca-file=/etc/kubernetes/pki/ca.pem \
--requestheader-allowed-names=aggregator \
--requestheader-extra-headers-prefix=X-Remote-Extra- \
--requestheader-group-headers=X-Remote-Group \
--requestheader-username-headers=X-Remote-User \
...然后分别重启所有master节点的apiserver
systemctl daemon-reload && systemctl restart kube-apiserver重建metrics-server
kubectl replace --force -f metrics-server.yaml等待一段时间后，回到dashboard，刷新后发现 CPU和内存信息都出来了
同时，安装完metrics-server后，我们也可以在kubelet中使用metrics-server,如：
kubectl top node
#NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
#kube-n-60-101.jieee.xyz 465m 11% 3503Mi 44%
#kube-n-60-102.jieee.xyz 257m 6% 2600Mi 33%
#kube-n-60-103.jieee.xyz 414m 10% 4092Mi 52%
证书配置
由于dashboard中使用了自签证书，导致chrome中无法访问，带来了一些不便，接下来我们为dashboard配置上证书
方式一：使用已有证书
先删除dashboard，然后修改配置文件
#删除
kubectl delete -f dashboard-deploy.yaml
#修改配置文件
vi dashboard-deploy.yaml
#找到以下内容，然后删除
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kubernetes-dashboard
type: Opaque创建证书，可以通过阿里云申请1年免费证书，或者通过Let’s Encrypt生成90天免费证书，建免费证书存放在$HOME/certs目录下，取名为tls.crt和tls.key。
#创建证书
kubectl create secret generic kubernetes-dashboard-certs --from-file=$HOME/certs -n kubernetes-dashboard
#重新部署dashboard
kubectl apply -f dashboard-deploy.yaml如此，证书就配置完成了
方式二：使用ingress
如果集群中已存在ingress，并且ingress配置了ssl（dashboard不支持http访问，所有必须支持ssl），那么可以用ingress卸载字签证书并替换成新证书。
ingress的部署可以查看Kubernetes使用Ingress nginx暴露服务并配置证书
配置文件：
cat > dashboard-ingress.yaml<<EOF
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: dashboard
namespace: kubernetes-dashboard
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true" # 强制跳转https
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" # 代理后端https
spec:
tls:
- hosts:
- '*.lingjie.tech'
secretName: lingjie-tech
rules:
- host: dashboard.lingjie.tech
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
EOF
#部署ingress
kubectl apply -f dashboard-ingress.yaml然后我们绑定一下host（将dashboard.lingjie.tech绑定到ingress的service ip），就能通过https访问了。