K8S Mater节点部署
1、部署Kubernetes API服务部署
- apiserver提供集群管理的REST API接口,包括认证授权、数据校验以及集群状态变更等。
- 只有API Server才能直接操作etcd;
- 其他模块通过API Server查询或修改数据
- 提供其他模块之间的数据交互和通信枢纽
(1)准备软件包
[root@linux-node1 ~]# cd /usr/local/src/kubernetes
[root@linux-node1 kubernetes]# cp server/bin/kube-apiserver /opt/kubernetes/bin/
[root@linux-node1 kubernetes]# cp server/bin/kube-controller-manager /opt/kubernetes/bin/
[root@linux-node1 kubernetes]# cp server/bin/kube-scheduler /opt/kubernetes/bin/ 只需要在linux-node1上拷贝
(2)创建生成CSR的 JSON 配置文件
[root@linux-node1 ~]# cd /usr/local/src/ssl
[root@linux-node1 ssl]# vim kubernetes-csr.json
{
"CN": "kubernetes",
"hosts": [
"127.0.0.1",
"192.168.56.110", #Master的ip地址
"10.1.0.1",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size":
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
(3)生成 kubernetes 证书和私钥
[root@linux-node1 ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
-ca-key=/opt/kubernetes/ssl/ca-key.pem \
-config=/opt/kubernetes/ssl/ca-config.json \
-profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
[root@linux-node1 ssl]# cp kubernetes*.pem /opt/kubernetes/ssl/
[root@linux-node1 ssl]# scp kubernetes*.pem 192.168.56.120:/opt/kubernetes/ssl/
[root@linux-node1 ssl]# scp kubernetes*.pem 192.168.56.130:/opt/kubernetes/ssl/
(4) 创建 kube-apiserver 使用的客户端 token 文件
[root@linux-node1 ~]# head -c /dev/urandom | od -An -t x | tr -d ' '
ad6d5bb607a186796d8861557df0d17f
[root@linux-node1 ~]# vim /opt/kubernetes/ssl/bootstrap-token.csv
ad6d5bb607a186796d8861557df0d17f,kubelet-bootstrap,,"system:kubelet-bootstrap"
(5) 创建基础用户名/密码认证配置
[root@linux-node1 ~]# vim /opt/kubernetes/ssl/basic-auth.csv
admin,admin,
readonly,readonly,
(6) 部署Kubernetes API Server
[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target [Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \
--bind-address=192.168.56.110 \
--insecure-bind-address=127.0.0.1 \
--authorization-mode=Node,RBAC \
--runtime-config=rbac.authorization.k8s.io/v1 \
--kubelet-https=true \
--anonymous-auth=false \
--basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
--service-cluster-ip-range=10.1.0.0/ \
--service-node-port-range=- \
--tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
--tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/kubernetes/ssl/ca.pem \
--etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
--etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
--etcd-servers=https://192.168.56.110:2379,https://192.168.56.120:2379,https://192.168.56.130:2379 \
--enable-swagger-ui=true \
--allow-privileged=true \
--audit-log-maxage= \
--audit-log-maxbackup= \
--audit-log-maxsize= \
--audit-log-path=/opt/kubernetes/log/api-audit.log \
--event-ttl=1h \
--v= \
--logtostderr=false \
--log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=
Type=notify
LimitNOFILE= [Install]
WantedBy=multi-user.target
(7) 启动API Server服务
[root@linux-node1 ~]# systemctl daemon-reload
[root@linux-node1 ~]# systemctl enable kube-apiserver
[root@linux-node1 ~]# systemctl start kube-apiserver
[root@linux-node1 ~]# systemctl status kube-apiserver
[root@linux-node1 ssl]# netstat -tulnp |grep kube-apiserver
tcp 192.168.56.110: 0.0.0.0:* LISTEN /kube-apiserver
tcp 127.0.0.1: 0.0.0.0:* LISTEN /kube-apiserver
从监听端口可以看到api-server监听在6443端口,同时也监听了本地的8080端口,是提供kube-schduler和kube-controller使用。
2、部署Controller Manager服务
- controller-manager由一系列的控制器组成,它通过apiserver监控整个集群的状态,并确保集群处于预期的工作状态。
(1)部署Controller Manager服务
[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service]
ExecStart=/opt/kubernetes/bin/kube-controller-manager \
--address=127.0.0.1 \
--master=http://127.0.0.1:8080 \
--allocate-node-cidrs=true \
--service-cluster-ip-range=10.1.0.0/ \
--cluster-cidr=10.2.0.0/ \
--cluster-name=kubernetes \
--cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \
--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \
--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \
--root-ca-file=/opt/kubernetes/ssl/ca.pem \
--leader-elect=true \
--v= \
--logtostderr=false \
--log-dir=/opt/kubernetes/log Restart=on-failure
RestartSec= [Install]
WantedBy=multi-user.target
(2)启动Controller Manager
[root@linux-node1 ~]# systemctl daemon-reload
[root@linux-node1 scripts]# systemctl enable kube-controller-manager
[root@linux-node1 scripts]# systemctl start kube-controller-manager
[root@linux-node1 scripts]# systemctl status kube-controller-manager
[root@linux-node1 ssl]# netstat -tulnp |grep kube-controlle
tcp 127.0.0.1: 0.0.0.0:* LISTEN /kube-controlle
从监听端口上,可以看到kube-controller监听在本地的10252端口,外部是无法直接访问kube-controller,需要通过api-server才能进行访问。
3、部署Kubernetes Scheduler
- scheduler负责分配调度Pod到集群内的node节点
- 监听kube-apiserver,查询还未分配的Node的Pod
- 根据调度策略为这些Pod分配节点
[root@linux-node1 ~]# vim /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service]
ExecStart=/opt/kubernetes/bin/kube-scheduler \
--address=127.0.0.1 \
--master=http://127.0.0.1:8080 \
--leader-elect=true \
--v= \
--logtostderr=false \
--log-dir=/opt/kubernetes/log Restart=on-failure
RestartSec= [Install]
WantedBy=multi-user.target
[root@linux-node1 ~]# systemctl daemon-reload
[root@linux-node1 scripts]# systemctl enable kube-scheduler
[root@linux-node1 scripts]# systemctl start kube-scheduler
[root@linux-node1 scripts]# systemctl status kube-scheduler
[root@linux-node1 ssl]# netstat -tulnp |grep kube-scheduler
tcp 127.0.0.1: 0.0.0.0:* LISTEN /kube-scheduler
从kube-scheduler的监听端口上,同样可以看到监听在本地的10251端口上,外部无法直接访问,同样是需要通过api-server进行访问。
4、部署kubectl 命令行工具
kubectl用于日常直接管理K8S集群,那么kubectl要进行管理k8s,就需要和k8s的组件进行通信,也就需要用到证书。此时kubectl需要单独部署,也是因为kubectl也是需要用到证书,而前面的kube-apiserver、kube-controller、kube-scheduler都是不需要用到证书,可以直接通过服务进行启动。
(1)准备二进制命令包
[root@linux-node1 ~]# cd /usr/local/src/kubernetes/client/bin
[root@linux-node1 bin]# cp kubectl /opt/kubernetes/bin/
(2)创建 admin 证书签名请求
[root@linux-node1 ~]# cd /usr/local/src/ssl/
[root@linux-node1 ssl]# vim admin-csr.json
{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size":
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "system:masters",
"OU": "System"
}
]
}
(3)生成 admin 证书和私钥
[root@linux-node1 ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
-ca-key=/opt/kubernetes/ssl/ca-key.pem \
-config=/opt/kubernetes/ssl/ca-config.json \
-profile=kubernetes admin-csr.json | cfssljson -bare admin
[root@linux-node1 ssl]# ls -l admin*
-rw-r--r-- root root Mar : admin.csr
-rw-r--r-- root root Mar : admin-csr.json
-rw------- root root Mar : admin-key.pem
-rw-r--r-- root root Mar : admin.pem [root@linux-node1 ssl]# cp admin*.pem /opt/kubernetes/ssl/
(4)设置集群参数
[root@linux-node1 ssl]# kubectl config set-cluster kubernetes \
--certificate-authority=/opt/kubernetes/ssl/ca.pem \
--embed-certs=true \
--server=https://192.168.56.110:6443
Cluster "kubernetes" set.
(5)设置客户端认证参数
[root@linux-node1 ssl]# kubectl config set-credentials admin \
--client-certificate=/opt/kubernetes/ssl/admin.pem \
--embed-certs=true \
--client-key=/opt/kubernetes/ssl/admin-key.pem
User "admin" set.
(6)设置上下文参数
[root@linux-node1 ssl]# kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=admin
Context "kubernetes" created.
(7)设置默认上下文
[root@linux-node1 src]# kubectl config use-context kubernetes
Switched to context "kubernetes".
上面(4)-->(7)的配置是为了在家目录下生成config文件,之后kubectl和api通信就需要用到该文件,这也就是说如果在其他节点上需要用到这个kubectl,就需要将该文件拷贝到其他节点。
[root@linux-node1 ~]# cat .kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR2akNDQXFhZ0F3SUJBZ0lVUUszc2dUb0lrNlhmVDhFaUVvcXNNc2pQQUVzd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJo
......
(8)使用kubectl工具
[root@linux-node1 ssl]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd- Healthy {"health": "true"}
etcd- Healthy {"health": "true"}
etcd- Healthy {"health": "true"}