10-kubernetes serveraccount RBAC

2024-03-13 12:12:33

认证安全
- serviceAccountName 和 userAccount
  - serviceaccount 创建
  - 使用admin 的SA
- 测试 URL访问kubernetes资源
APIserver客户端定义的配置文件
- kubernetes 集群相关的私有CA证书
- 创建新的apiserver的账号及证书

认证安全

任何用途操作集群的资源对象是,都要经历三种安全相关的操作:

任何用户来访问时, 都需要完成kubernetes系统认证操作
认证通过后, 进行授权检查
准入控制, 检查是否有权限操作其它的一些资源操作

认证方式:

令牌认证
SSL 秘钥认证, 也是最常用的方式.
RBAC 全程:Role Base AccessControl 授权检查机制

客户端 --> API Server 传递的参数.

user: username, uid
group:
extra:

API
Request path:
    /apis/apps/v1/namespaces/default/deployments/myapp-deploy/

请求动作 HTTP request verb:
    get  post  pu  delete

映射到kubernetes请求的动作:
    get  list  create  update patch watch proxy  redirect delete  deletecollection

请求访问的资源 Resource

请求的子资源 Subresource

API group

serviceAccountName 和 userAccount

kubernetes 有两类认证值的用户账号，分别是

serviceAccountName: 是用于Pod客户端认证使用的账户
userAccount: 用于真实用户，或者说操作集群资源的人使用的账户

serviceaccount 创建

可以使用kubectl create serviceaccount USER_NAME 来创建

[root@master manifests]# kubectl create serviceaccount admin
serviceaccount/admin created
[root@master manifests]# kubectl get sa
NAME           SECRETS   AGE
admin          1         2s
default        1         37d
[root@master manifests]# kubectl describe sa admin
Name:                admin
Namespace:           default
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   admin-token-48wdj
Tokens:              admin-token-48wdj      # 自动生产的token
Events:              <none>

当创建一个sa后，会自动创建一个secret

[root@master manifests]# kubectl get secret
NAME                       TYPE                                  DATA   AGE
admin-token-48wdj          kubernetes.io/service-account-token   3      3m3s        # 这里自动创建的是认证信息，但不代表有权限。
default-token-bc86p        kubernetes.io/service-account-token   3      37d
[root@master manifests]# kubectl describe secret admin-token-48wdj
Name:         admin-token-48wdj
Namespace:    default
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin
              kubernetes.io/service-account.uid: dc343339-8eaf-4022-a03a-4cc8df8c9ebf

Type:  kubernetes.io/service-account-token

Data
====
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLTQ4d2RqIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZGMzNDMzMzktOGVhZi00MDIyLWEwM2EtNGNjOGRmOGM5ZWJmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xgehKowaM6sHykd6vbfi4hu4M0DxFAYrfuY9fkN_8vkcpi_c4HC8yl0MZD4GQivRVyEPb_7X6A81BkaHLVEHvbt7bl20Gk5F2tnFxWbSGMGNh2pRa0GxtnWEO03EyGHt1Yl_YGdqT13UnwfUFC5302P7dvuC15OKjd3az-vaMu3YgUkAKyceZZRltBasmoWVHdAY1u2kVFpjT60TYPtAyV6eAeDaucB94Ye60EmyhAHBNP8DkdFbWcHT25rdRQa72B_IVZ_ZaeCfJtZkpsg4XplniC1OuXV_25nqXCNRbIqp_yaH7dT0NKhT9_EYEC4b8MYZcL2e_RZyNgBWSUf-9Q
ca.crt:     1025 bytes
namespace:  7 bytes

使用admin 的SA

定义一个清单文件，使用admin的SA

[root@master manifests]# cat pod-sa-demo.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-sa-demo
  namespace: default
  labels:
    app: myapp
    tier: frontend
  annotations:
    jubaozhu.com/created-by: "cluster admin"
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
  serviceAccountName: admin
 [root@master manifests]# kubectl apply -f pod-sa-demo.yaml 
pod/pod-sa-demo created
[root@master manifests]# kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
pod-sa-demo   1/1     Running   0          3s

查看该Pods的详细信息中servername

[root@master manifests]# kubectl describe pods pod-sa-demo | grep SecretName
    SecretName:  admin-token-48wdj  # 这里和自动升昌的admin对应自动生成的secret名称是一样的。

也可以通过--dry-run 和 -o yaml 来获取yaml配置文件

[root@master manifests]# kubectl create serviceaccount admin --dry-run -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: null
  name: admin

这样就能把导出的yaml格式的保存到文件中，在使用kubectl apply 来执行。

测试 URL访问kubernetes资源

首先，本地要启动一个proxy代理，之后通过这个代理来访问

[root@master ~]# kubectl proxy --port=8080
Starting to serve on 127.0.0.1:8080

再起一个新的终端进行测试

[root@master pki]# curl http://localhost:8080/apis/apps/v1/namespaces/kube-system/deployments/coredns
{
  "kind": "Deployment",
  "apiVersion": "apps/v1",
  "metadata": {
    "name": "coredns",
    "namespace": "kube-system",
    "selfLink": "/apis/apps/v1/namespaces/kube-system/deployments/coredns",
    "uid": "c730eb98-8fe7-47a8-9856-1fc3ff2044aa",
    "resourceVersion": "1039",
    "generation": 1,
    "creationTimestamp": "2019-07-09T08:36:50Z",
    "labels": {
      "k8s-app": "kube-dns"
    },
    "annotations": {
      "deployment.kubernetes.io/revision": "1"
    }
  },
  "spec": {
    "replicas": 2,
    "selector": {
      "matchLabels": {
        "k8s-app": "kube-dns"
... ...
... ...

/apis/apps/v1/namespaces/kube-system/deployments/coredns 这里的访问路径都是固定套路。

这里的路径区别

以 api 其实的资源，均为核心资源

剩下的其他资源，都需要以 apis 为起始路径。

有哪些资源的客户端需要和APIserver有交互

[root@master pki]# kubectl config set-cluster mycluster --kubeconfig=/tmp/test.cnf --server="https://10.0.20.20:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "mycluster" set.
[root@master pki]# kubectl config view --kubeconfig=/tmp/test.cnf 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.0.20.20:6443
  name: mycluster
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []

授权插件:

Node
ABAC
RBAC
Webhook http的回调机制

角色机制: 给特定的角色赋权, 在把对应的角色赋予给用户, 那么该用户就拥有该角色的权限.

APIserver客户端定义的配置文件

查看config命令帮助：

[root@master manifests]# kubectl config --help
Modify kubeconfig files using subcommands like "kubectl config set current-context my-context"

 The loading order follows these rules:

  1.  If the --kubeconfig flag is set, then only that file is loaded. The flag may only be set once and no merging takes
place.
  2.  If $KUBECONFIG environment variable is set, then it is used as a list of paths (normal path delimiting rules for
your system). These paths are merged. When a value is modified, it is modified in the file that defines the stanza. When
a value is created, it is created in the first file that exists. If no files in the chain exist, then it creates the
last file in the list.
  3.  Otherwise, ${HOME}/.kube/config is used and no merging takes place.

Available Commands:
  current-context Displays the current-context
  delete-cluster  Delete the specified cluster from the kubeconfig
  delete-context  Delete the specified context from the kubeconfig
  get-clusters    Display clusters defined in the kubeconfig
  get-contexts    Describe one or many contexts
  rename-context  Renames a context from the kubeconfig file.
  set             Sets an individual value in a kubeconfig file
  set-cluster     Sets a cluster entry in kubeconfig
  set-context     Sets a context entry in kubeconfig
  set-credentials Sets a user entry in kubeconfig
  unset           Unsets an individual value in a kubeconfig file
  use-context     Sets the current-context in a kubeconfig file
  view            Display merged kubeconfig settings or a specified kubeconfig file

Usage:
  kubectl config SUBCOMMAND [options]

Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

名称	注解
set-cluster	设定集群
set-credentials	设定用户账号
set-context	设定上下文
use-context	设定当前上下文

可以直接通过命令的方式进行创建，也可以使用配置清单来定义后，使用kubectl apply -f 来执行设定；

查看当前配置文件的示例说明

[root@master manifests]# kubectl config view
apiVersion: v1                                      # 这里可以看出，config也是一个标准的kubernetes资源对象
clusters:                                           # 集群列表
- cluster:                                          # 定义一个集群，在下面可以定义多个
    certificate-authority-data: DATA+OMITTED        # 服务器发过来的认证方式
    server: https://10.0.20.20:6443                 # 访问验证地址
  name: kubernetes                                  # 集群名称
contexts:                                           # 上下文列表
- context: 
    cluster: kubernetes                             # 对应用户能访问的集群名称
    user: kubernetes-admin                          # 该上下文对应的用户
  name: kubernetes-admin@kubernetes                 # 此上下文的名称
current-context: kubernetes-admin@kubernetes        # 当前上下文
kind: Config
preferences: {}
users:                                              # 用户列表
- name: kubernetes-admin                            # 用户已名称
  user:
    client-certificate-data: REDACTED               # 客户端的认证方式
    client-key-data: REDACTED                       # key信息

kubernetes 集群相关的私有CA证书

[root@master ~]# ll /etc/kubernetes/pki/
-rw-r--r-- 1 root root 1233 Jul  9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul  9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul  9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul  9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul  9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul  9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul  9 16:36 ca.crt
-rw------- 1 root root 1679 Jul  9 16:36 ca.key
drwxr-xr-x 2 root root  162 Jul  9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul  9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul  9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul  9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul  9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul  9 16:36 sa.key
-rw------- 1 root root  451 Jul  9 16:36 sa.pub

创建新的apiserver的账号及证书

新建的密钥对就直接放到kubernetes集群对应的pki目录下

[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# ll
total 60
-rw-r--r-- 1 root root 1233 Jul  9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul  9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul  9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul  9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul  9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul  9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul  9 16:36 ca.crt
-rw------- 1 root root 1679 Jul  9 16:36 ca.key
drwxr-xr-x 2 root root  162 Jul  9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul  9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul  9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul  9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul  9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul  9 16:36 sa.key
-rw------- 1 root root  451 Jul  9 16:36 sa.pub

创建私钥

[root@master pki]# (umask 077; openssl genrsa -out tracy.key 2048)
Generating RSA private key, 2048 bit long modulus
.........................+++
.....................................................+++
e is 65537 (0x10001)
[root@master pki]# ll tracy.key 
-rw------- 1 root root 1679 Aug 19 16:07 tracy.key

然后基于tracy.key私钥来生成证书使用kubernetes集群的ca.crt来签署

生成证书签署请求

[root@master pki]# openssl req -new -key tracy.key -out tracy.csr -subj "/CN=tracy"
# 注意，最后的 "/CN=tracy" 中的tracy，就是用户名

用ca.crt 来签署

[root@master pki]# openssl x509 -req -in tracy.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tracy.csr -days 365
Signature ok
subject=/CN=tracy
Getting CA Private Key
[root@master pki]# ll tracy.*
-rw-r--r-- 1 root root  973 Aug 19 16:14 tracy.csr
-rw------- 1 root root 1679 Aug 19 16:07 tracy.key

验证查看生成的证书

[root@master pki]# openssl x509 -in tracy.csr -text -noout
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            ea:48:9e:47:1b:2e:19:ac
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes           # 这里可以看到是kubernetes自己的ca.crt证书签署的
        Validity
            Not Before: Aug 19 08:14:53 2019 GMT        # 证书有效期限起始
            Not After : Aug 18 08:14:53 2020 GMT        # 证书有效期限结束
        Subject: CN=tracy       # 作为用户账号连入kubernetes用户
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:cd:e3:a1:7a:db:17:70:57:16:b3:f6:6a:6c:87:
                    50:5f:2f:38:86:67:93:fc:b2:ba:5f:c4:60:16:66:
                    2f:67:29:01:c7:f0:b5:d8:ca:c6:db:ca:42:8a:a8:
                    ce:17:ac:d1:43:64:eb:a8:1b:34:6c:97:d9:49:dc:
                    24:b9:ab:8c:cd:be:6a:ad:ff:70:64:40:51:40:0f:
                    16:d5:61:73:dc:3f:aa:57:4f:7a:c8:60:a7:37:ee:
                    53:d0:ec:25:78:fc:4a:af:81:67:2c:d8:08:0f:a0:
                    4d:38:0f:28:36:4b:f7:c6:bb:a2:0f:4b:33:e4:77:
                    44:83:4d:14:2e:ef:ce:d4:6a:47:37:09:0b:9d:41:
                    ad:07:62:8e:06:94:71:04:f7:1b:a2:e7:6a:d6:86:
                    8e:7c:45:69:5e:a2:74:22:6a:8f:47:c8:ef:ba:f7:
                    d4:08:05:6e:c0:ae:36:bc:68:4b:7f:a5:c8:1c:87:
                    c4:34:a7:28:94:96:49:b1:cc:53:c0:75:7b:c9:44:
                    f4:78:c9:f4:fa:e2:5a:55:64:0d:dd:0a:40:31:19:
                    4e:80:f2:cf:02:f4:48:ae:65:93:a9:e7:41:d7:0b:
                    e1:7a:27:38:dc:4f:9d:12:be:8e:62:2b:73:98:0c:
                    45:07:c5:ad:43:91:06:0e:30:4c:5c:63:9d:25:de:
                    bb:29
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha256WithRSAEncryption
         74:7b:c9:fd:9e:2d:4e:84:c1:f8:83:8d:9a:37:c1:50:27:e3:
         f6:1c:d4:ea:ae:4f:83:f3:dc:23:81:01:fd:3d:02:e1:26:7c:
         8a:94:a7:dd:0a:3c:86:52:45:2a:25:b3:63:3f:51:15:87:0f:
         f9:25:8b:eb:b0:7e:0f:7c:4a:f1:23:b9:72:7e:c6:87:55:16:
         7a:bf:29:2e:da:5f:3c:4c:f2:b2:77:09:fb:fe:5a:38:67:1b:
         db:c5:db:4b:64:87:1c:c7:e8:71:84:92:fd:39:17:9d:7b:b3:
         dc:4b:1a:56:a8:75:63:1e:be:80:ac:98:a6:d7:41:96:8d:44:
         b4:13:79:c6:08:07:b8:4a:8c:5b:04:09:a6:8d:73:47:b5:a8:
         0b:97:40:66:cf:04:bc:01:93:4a:6a:ed:b2:91:94:51:12:0e:
         dd:c9:70:c9:fd:65:ed:49:c2:1a:89:0f:6e:ed:69:1c:d5:2c:
         cb:42:f4:ce:69:bd:99:a2:ff:ea:a7:69:5f:f7:43:b5:c3:03:
         86:67:95:6e:1e:80:c7:8c:35:ac:7f:e6:e2:4e:11:08:dc:cd:
         16:02:e1:cd:59:4d:70:e8:fe:5a:85:4a:18:97:bb:97:f4:6a:
         82:ee:e3:c1:73:b4:2a:24:73:bd:f1:fc:08:02:6c:5d:e1:53:

设定用户账号

[root@master pki]# kubectl config set-credentials tracy --client-certificate=tracy.csr --client-key=tracy.key --embed-certs=true
User "tracy" set.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.0.20.20:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: tracy           # 这里可以看到刚刚设定的用户账号
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

tracy加入上下文

[root@master pki]# kubectl config set-context tracy@kubernetes --cluster=kubernetes --user=tracy
Context "tracy@kubernetes" created.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.0.20.20:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
- context:                          # 这里开始向下就是tracy的上下文相关的配置了
    cluster: kubernetes
    user: tracy
  name: tracy@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: tracy
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

应用刚刚增加的tracy的对应的上下文

[root@master pki]# kubectl config use-context tracy@kubernetes
Switched to context "tracy@kubernetes".
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.0.20.20:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
- context:
    cluster: kubernetes
    user: tracy
  name: tracy@kubernetes
current-context: tracy@kubernetes       # 看到当前上下文已经切换到 tracy@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: tracy
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

测试

由于tracy@kubernetes 没有赋予任何角色，所以对集群是没有任何操作权限

[root@master pki]# kubectl get pods
Error from server (Forbidden): pods is forbidden: User "tracy" cannot list resource "pods" in API group "" in the namespace "default"

这里报错意思是 default 名称空间中，不能获取数据

重新切换至kubernetes-admin@kubernetes测试

[root@master pki]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[root@master pki]# kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
pod-sa-demo   1/1     Running   0          3d3h

保存配置文件

默认 kubernetes-admin 的配置文件保存目录是~/.kube/config文件中。

也可以保存到别的目录下：

[root@master ~]# kubectl config set-cluster test --kubeconfig=/tmp/test.conf --server="https://10.0.20.20:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "test" set.
[root@master ~]# kubectl config view --kubeconfig=/tmp/test.conf 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.0.20.20:6443
  name: test
contexts: []

码农公寓