k8s基础知识

3、k8s核心技术

3.5、kubectl命令行工具

kubectl [action] [type] [name] [option]

  • action:动作,如create、get、apply、delete、logs、describe。动作可以不是一个单词,而是多个,如set image
  • type:资源类型、如 pod,nodes
  • name:资源名称,如abcmaster
  • option:选项,如-n

3.6、Pod

Pod实现机制

  • 共享网络
    • Pod中的容器在根容器(Pause)的桥接下形成同一个namespace
  • 共享存储
    • 共用Volumn数据卷

Pod镜像拉取策略与重启策略

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: nginx
    image: nginx:1.14
    imagePullPolicy: Always
  restartPolicy: Never

imagePullPolicy 镜像拉取策略

  • IfNotPresent:默认值,宿主机下不存在时拉取
  • Always:每次创建pod都重新拉取
  • Never:永远不主动拉取

restartPolicy 容器重启策略

  • Always:总是重启
  • onFailure:异常退出时(退出码非0)
  • Never:从不主动重启

Pod的资源限制

apiVersion: v1
kind: Pod
metadata:
  name: mysql
spec:
  containters:
  - name: mysql
    image: mysql
    env:
    - name: MYSQL_ROOT_PASSWARD
      value: "123456"
    
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
        
      limits:
        memory: "128Mi"
        cpu: "500m"

resources Pod中容器资源限制

  • requests:最小资源要求
  • limits:最大给予资源限制

其中:cpu 1c == 1000m

Pod的节点调度策略

  • 1、容器资源限制
  • 2、nodeSelector 标签选择器
  • 3、nodeAffinity 节点亲和性选择器(更灵活的选择器)
    • 硬亲和性(必须满足)
    • 软亲和性(尽量满足)
    • 操作符:In、NotIn,Gt、Lt、Exist …

3.7、Controller

3.7.1、Pod与Controller的关系

  • 控制器专指的是Pod的控制器
  • Pod通过Controller来实现应用的运维,如伸缩,回滚,滚动升级等。
  • Controller使用label(selector)与要管理的Pod建立联系

3.7.2、deployment控制器

作用:

  • 部署无状态应用
  • 管理Pod和ReplicaSet
  • 部署,滚动升级

yaml配置说明

  • 使用 --dry-run命令生成一个deployment的yaml文件
root@ubuntu:~# kubectl create deployment web --image=nginx --dry-run -o yaml > web.yaml
W0919 06:22:09.514872   82865 helpers.go:555] --dry-run is deprecated and can be replaced with --dry-run=client.
root@ubuntu:~# ls
kube-flannel.yml  kubernetes-dashboard.yaml  pullk8s.sh  snap  web.yaml

查看生成的web.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: web
  name: web
spec:
  replicas: 1  # 副本数
  selector:
    matchLabels: # 使用label选择器
      app: web
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:  # 定义label标签
        app: web
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}
  • 使用yaml部署deployment
root@ubuntu:~# kubectl apply -f web.yaml
deployment.apps/web created

root@ubuntu:~# kubectl get pods
NAME                  READY   STATUS    RESTARTS   AGE
web-96d5df5c8-n7ht9   1/1     Running   0          39s
  • 使用yaml对外发布使用yaml创建的deployment
# 生成发布用yaml
root@ubuntu:~# kubectl expose deployment web --port=80 --type=NodePort --target-port=80 --name=nginx1 -o yaml > web1.yaml
# 执行发布
root@ubuntu:~# kubectl apply -f web1.yaml
Warning: resource services/nginx1 is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
service/nginx1 configured

查看生成的web1.yaml

apiVersion: v1
kind: Service # 可以看到expose生成的是一个service,所以可以从外部访问
metadata:
  creationTimestamp: "2021-09-19T06:36:32Z"
  labels:
    app: web
  name: nginx1
  namespace: default
  resourceVersion: "66025"
  uid: b56685c8-6505-4c15-92a7-031f6d5df9a5
spec:
  clusterIP: 10.109.236.204
  clusterIPs:
  - 10.109.236.204
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 31150
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: web
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

应用升级回滚和弹性伸缩

  • 使用set image命令执行deployment的应用升级
kubectl set image deploment web nginx=nginx:1.15
  • 查看历史版本
kubectl rollout history deployment web
  • 回滚到上一个版本
kubectl rollout undo deployment web
  • 使用scale弹性伸缩命令创建5个副本
kubectl scale deployment web --replicas=5

3.7.3、Service

  • Service用于定义一组pod的访问规则,相当于微服务的注册中心,在kubectl命令中简称svc
  • 1、通过服务发现机制防止pod失联。因为pod的ip是不固定的,pod将自己的ip注册到Service中。
  • 2、定义Pod的访问策略(负载均衡)

Service与Pod的关系

  • 与Controller相同,也是通过selector – labels 建立关系

常用Service类型

Service使用expose命令创建,其中的 --type 用于定义Service的类型,一共有以下类型:

  • ClusterIP:供集群内部访问
  • NodePort:对外暴露应用
  • LoadBalancer:对外暴露应用,用于公有云

3.7.4、有状态Controller – StatefulSet

有状态Controller的特点

  • 各个Pod有启动顺序要求
  • 可能指定所在node(固定ip)
  • 每个pod独立的,有唯一的网络标识符,不能随意伸缩和拓展
  • 持久存储

部署有状态控制器

***无头Service:***ClusterIP的值为none,使用特定域名访问的Service。

3.7.5、DaemonSet 守护进程

  • 确保所有node都运行同一个pod,如在每一个node中都部署一个数据采集工具
# 进入某个pod内部
kubectl exec -it ${podname} bash
# 退出
exit

3.7.6、Job与Cronjob

  • 一次性任务与周期任务

3.8、配置管理

3.8.1、Secret

  • 产生加密数据并存储在etcd中,使Pod可以以Volume的形式访问使用
  • 使用场景:保存登陆凭证

创建Secret

  • 编辑yaml文件
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  username: YWRtaW4=
  password: MTIzNDU2
  • 使用yaml创建Secret
root@ubuntu:~# kubectl create -f mysecret.yaml
secret/mysecret created
root@ubuntu:~# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-pxtlb   kubernetes.io/service-account-token   3      13d
mysecret              Opaque                                2      22s

以变量模式挂载到pod中

  • 创建使用了secret的Pod
apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: sng
    image: nginx
    env:
    - name: SECRET_USERNAME
      valueFrom:
        secretKeyRef:
          name: mysecret
          key: username
    - name: SECRET_PASSWORD
      valueFrom:
        secretKeyRef:
          name: mysecret
          key: password
  • 部署Pod并查看变量信息
# 部署
root@ubuntu:~# kubectl apply -f sng.yaml
pod/mypod created
root@ubuntu:~# kubectl get pods
NAME                  READY   STATUS              RESTARTS   AGE
mypod                 0/1     ContainerCreating   0          15s
web-96d5df5c8-n7ht9   1/1     Running             0          117m
root@ubuntu:~# kubectl get pods
NAME                  READY   STATUS    RESTARTS   AGE
mypod                 1/1     Running   0          25s
web-96d5df5c8-n7ht9   1/1     Running   0          117m

# 进入pod内部
root@ubuntu:~# kubectl exec -it mypod bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
# 输出变量
root@mypod:/# echo $SECRET_USERNAME
admin

使用Volume方式使用

  • 创建使用了Volume的pod
apiVersion: v1
kind: Pod
metadata:
  name: vng
spec:
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: foo
      mountPath: "/etc/foo"   # 容器内部的挂载点
      readOnly: true
  volumes:
  - name: foo
    secret:
      secretName: mysecret
  • 创建Pod并查看挂载的secret
root@ubuntu:~# kubectl apply -f vng.yaml
pod/vng created
root@ubuntu:~# kubectl get pods
NAME                  READY   STATUS    RESTARTS   AGE
mypod                 1/1     Running   0          15m
vng                   1/1     Running   0          23s
web-96d5df5c8-n7ht9   1/1     Running   0          132m
root@ubuntu:~# kubectl exec -it vng bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
# 查看挂载效果
root@vng:/# ls
bin   docker-entrypoint.d   home   media  proc  sbin  tmp
boot  docker-entrypoint.sh  lib    mnt    root  srv   usr
dev   etc                   lib64  opt    run   sys   var
root@vng:/# ls /etc/foo
password  username
root@vng:/# cat /etc/foo/username
admin

3.8.2、ConfigMap

  • 存储不加密数据到etcd中,使pod可以变量或数据卷方式使用,与secret功能类似
  • 常用于生成应用的配置文件,如Redis.properties

创建ConfigMap

  • 创建一个配置文件redis.properties
redis.host=127.0.0.1
redis.port=7397
redis.password=123456
  • 使用create命令创建
# 创建
kubectl create configmap redis_conf --from-file=redis.properties
# 查看
kubectl get cm

3.9、集群安全机制

资源访问3步走:

  • 认证
  • 鉴权
  • 准入控制

3.9.1、安全机制实现步骤

常用认证方式

  • 基于ca证书的https认证
  • token
  • 基于http的用户名 + 密码

鉴权(授权)实现模式

  • 基于RBAC的权限体系

准入控制

  • 如果准入控制列表中存在访问的资源,则通过,否则拒绝

3.9.1、基于RBAC的鉴权控制

  • 角色
    • Role:针对特定namespace的访问权限
    • ClusterRole: 针对所有ns的访问权限
  • 权限主体(subject)
    • user
    • group
    • serviceaccount
  • 资源(resources)
    • namespace
    • pod
    • node
  • 角色绑定
    • roleBinding:将角色绑定到主体
    • clusterRoleBinding:将集群角色绑定给主体

创建角色

  • 创建一个命名空间
root@ubuntu:~# kubectl create ns roledemo
namespace/roledemo created
  • 在roledemo中创建一个pod
root@ubuntu:~# kubectl run nginx --image=nginx -n roledemo
pod/nginx created
root@ubuntu:~# kubectl get pods -n roledemo
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          22s
  • 使用yaml定义Role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: roledemo
  name: pod-reader
rules:
- apiGroups: [""]  # 最前面的 - 横杠表示下面是数组中的一个元素
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
  • 执行创建
# 创建
root@ubuntu:~# kubectl apply -f rbac-role.yaml
role.rbac.authorization.k8s.io/pod-reader created

# 查询
root@ubuntu:~# kubectl get roles -n roledemo
NAME         CREATED AT
pod-reader   2021-09-19T11:34:24Z

创建roleBinding

  • 使用yaml定义角色绑定
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: roledemo
  name: read-pods
subjects:
- kind: User
  name: arsiya
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
  • 执行创建
root@ubuntu:~# kubectl apply -f rb.yaml
rolebinding.rbac.authorization.k8s.io/read-pods created

root@ubuntu:~# kubectl get rolebindings -n roledemo
NAME        ROLE              AGE
read-pods   Role/pod-reader   31s

创建证书

  • 编写证书脚本
cat > arsiya-csr.json <<EOF
{
    "CN": "arsiya",
    "hosts": [],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [{
        "C": "CN",
        "L": "Beijing",
        "ST": "Beijing"
    }]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes arsiya-csr.json | cfssljson -bare arsiya

kubectl config set-cluster kubernetes --certificate-authority=ca.pem --embed-certs=true --server=https://192.168.18.11:6443 --kubeconfig=arsiya-kubeconfig

kubectl config set-credentials arsiya --client-key=arsiya-key.pem --client-certificate=arsiya.pem --embed-certs=true --kubeconfig=arsiya-kubeconfig

kubectl config set-context default --cluster=kubernetes --user=arsiya --kubeconfig=arsiya-kubeconfig

kubectl config use-context default --kubeconfig=arsiya-kubeconfig

3.10、Ingress

Ingress相当于网关,可以作为统一的入口访问多个service关联的Pods。Ingress不是k8s的内置组件,需要单独安装。

  • Service的NodePort可以暴露内部服务,但NodePort在每一个Node上都会占用同一个Port。
  • 使用Ingress可以通过域名访问pod中的应用。

3.10.1、部署Ingress Controllor

  • 使用yaml定义Ingress控制器
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: lizhenliang/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container
    
  • 执行创建命令
root@ubuntu:~# kubectl apply -f ingress-controller.yaml

3.13、java项目的部署

3.13.1、使用Dockerfile制作镜像

  • 定义Dockerfile
FROM adoptopenjdk/openjdk8:latest
VOLUME /tmp
ADD ./demo2-0.0.1-SNAPSHOT.jar /demo2.jar
ENTRYPOINT ["java","-jar","/demo2.jar","&"]
  • 构建镜像
root@ubuntu:~# docker build -t demo2:latest .
Sending build context to Docker daemon  36.65MB
Step 1/4 : FROM adoptopenjdk/openjdk8:latest
 ---> 6331f760afd4
Step 2/4 : VOLUME /tmp
 ---> Running in ed378ef2df18
Removing intermediate container ed378ef2df18
 ---> 0737cfe3e07c
Step 3/4 : ADD ./demo2-0.0.1-SNAPSHOT.jar /demo2.jar
 ---> 955e0a724698
Step 4/4 : ENTRYPOINT ["java","-jar","/demo2.jar","&"]
 ---> Running in c364d268ffd1
Removing intermediate container c364d268ffd1
 ---> b39f955ca2cc
Successfully built b39f955ca2cc
Successfully tagged demo2:latest
  • 测试镜像
root@ubuntu:~# docker run -d -p 8089:8089 demo2:latest -t
d069b5822920d2c3d32ad53a15bdbc3436e580c061a2431d67b3c9f26e403db3

3.13.2、使用yaml创建Deployment

  • 创建yaml
root@ubuntu:~# kubectl create deployment demo2 --image=demo2:latest --dry-run -o yaml > demo2.yaml
W0920 11:07:09.196773  137211 helpers.go:555] --dry-run is deprecated and can be replaced with --dry-run=client.
上一篇:k8s 部署 Ingress 及使用


下一篇:ingress上传大小限制问题配置