基于eBPF的Kubernetes一站式监控平台【动手实践】

2022-12-09 20:06:07

背景介绍

本实验通过在ACK Pro集群上安装Kubernetes监控，体验无侵入地监控应用和查看服务拓扑。

相关参考文章：
1、什么是阿里云Kubernetes监控：https://help.aliyun.com/document_detail/260777.html

前提条件

创建 ACK 集群，请参见创建Kubernetes托管版集群。

操作步骤

步骤一接入Prometheus监控和Kubernetes监控

1、进入ARMS Kubernetes监控控制台（注意选region），点击安装

2、分别点击安装Prometheus监控和Kubernetes监控

3、点击创建

步骤二接入demo应用

1、demo应用是一个简单的电商应用：

多语言微服务应用：Java应用、NodeJS应用、Golang应用
中间件：Redis、MySQL、RocketMQ
网络调用协议：HTTP、Dubbo、Redis、MySQL

2、demo整体页面效果和架构图（通过K8S监控架构感知）：

3、开始动手上传并安装yaml，yaml在最后附录中：

4、确认安装成功，容器都正常启动，页面访问正常，否则提问

查看外部ip看下商城效果：

浏览器打开????端点IP，开始买买买！

步骤三探索集群概览和服务详情

查看异常应用，应用黄金指标，下钻查看应用Pod、容器详情等

步骤四查看集群拓扑

打开集群拓扑。查看系统全局架构，分析上下游调用关系，查看链路上应用黄金指标、网络等

其他探索：

1、查看拓扑调用关系/【边】

2、告警管理

总结

Kubernetes监控的接入非常简单，支持无侵入地监控应用黄金指标和网络性能指标，服务拓扑能全局地展示系统架构，方便做架构分析和上下游链路分析。感兴趣同学请扫码进一步了解：

附录

demo yaml

# Source: arms-demo/templates/configuration.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: configuration
  namespace: arms-apm-demo
data:
#  慢查询是否打开
  SLOW_QUERY: 'false'
---
# Source: arms-demo/templates/mysql-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
  namespace: arms-apm-demo
data:
  my.cnf: |
    [client]
    default-character-set=utf8mb4
    [mysql]
    default-character-set=utf8mb4
    [mysqld]
    character-set-server = utf8mb4
    collation-server = utf8mb4_unicode_ci
    init_connect='SET NAMES utf8mb4'
    skip-character-set-client-handshake = true
    max_connections=2000
    secure_file_priv=/var/lib/mysql
    bind-address=0.0.0.0
    symbolic-links=0
    sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
    wait_timeout=31536000
    interactive_timeout=31536000
    max_allowed_packet=500M
  initsql.sh: |
    #!/bin/bash
    #sleep 1000
    #mysql -u root -p cart@1234 -D cart_db </usr/local/work/order_data.sql
---
# Source: arms-demo/templates/barservice-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: userservice
  namespace: arms-apm-demo
spec:
  type: ClusterIP
  selector:
    app: userservice
  ports:
  - name: http
    port: 3000
    targetPort: 3000
---
# Source: arms-demo/templates/checkoutservice-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: checkoutservice
  namespace: arms-apm-demo
spec:
  type: ClusterIP
  selector:
    app: checkoutservice
  ports:
  - name: http
    port: 8089
    targetPort: 8089
---
# Source: arms-demo/templates/frontend-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: arms-apm-demo
spec:
  type: ClusterIP
  selector:
    app: frontend
  ports:
  - name: http
    port: 8080
    targetPort: 8080
---
# Source: arms-demo/templates/frontend-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: frontend-external-1
  namespace: arms-apm-demo
spec:
  type: LoadBalancer
  selector:
    app: frontend
  ports:
    - name: http
      port: 8080
      targetPort: 8080
---
# Source: arms-demo/templates/mysql.yaml
apiVersion: v1
kind: Service
metadata:
  name: "mysql-pod"
  namespace: arms-apm-demo
spec:
  type: NodePort
  selector:
    app: "mysql-pod"
  ports:
    - port: 3306
      targetPort: 3306
---
# Source: arms-demo/templates/nacos-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nacos-standalone
  namespace: arms-apm-demo
spec:
  type: ClusterIP
  selector:
    app: nacos-standalone
  ports:
  - name: http
    port: 8848
    targetPort: 8848
---
# Source: arms-demo/templates/redis-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: arms-apm-demo
spec:
  type: NodePort
  ports:
    - port: 6379
      nodePort: 30003
  selector:
    app: redis
---
# Source: arms-demo/templates/rocketmq-namesrv.yaml
apiVersion: v1
kind: Service
metadata:
  name: mqnamesrv
  namespace: arms-apm-demo
spec:
  ports:
    - nodePort: 31002
      port: 9876
      protocol: TCP
      targetPort: 9876
  selector:
    app: mqnamesrv
  type: NodePort
---
# Source: arms-demo/templates/barservice-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: userservice
  namespace: arms-apm-demo
spec:
  selector:
    matchLabels:
      app: userservice
  template:
    metadata:
      labels:
        app: userservice
    spec:
      containers:
        - name: userservice
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/barservice:v2
          imagePullPolicy: Always
          ports:
          - containerPort: 3000
          env:
          - name: SW_AGENT_NAME
            value: userservice
          - name: SW_AGENT_AUTHENTICATION
            value: adapt_aokcdqn3ly@03cd8c21942bee9_aokcdqn3ly@53df7ad2afe8301
          - name: SW_AGENT_COLLECTOR_BACKEND_SERVICES
            value: tracing-analysis-dc-hz-internal.aliyuncs.com:8000
---
# Source: arms-demo/templates/cartservice-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cartservice
  namespace: arms-apm-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cartservice
  template:
    metadata:
      labels:
        app: cartservice
      annotations:
        armsPilotAutoEnable: "on"
        armsPilotCreateAppName: "cartservice"
    spec:
      containers:
        - name: cartservice
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/cartservice:v2
          imagePullPolicy: Always
          env:
          - name: dubbo.registry.address
            value: "nacos://nacos-standalone:8848"
          - name: spring.cloud.nacos.discovery.server-addr
            value: "nacos-standalone:8848"
          - name: spring.redis.host
            value: "redis"
          - name: spring.redis.password
            value: ""
          - name: spring.datasource.url
            value: "jdbc:mysql://mysql-pod:3306/cart_db?characterEncoding=utf-8&useSSL=false&autoReconnect=true"
          - name: spring.datasource.username
            value: "root"
          - name: spring.datasource.password
            value: cart@1234
          - name: rocketmq.alimq.enable
            value: "false"
          - name: rocketmq.nameSrvAddr
            value: "http://1588027496779518.mqrest.cn-shanghai.aliyuncs.com"
          - name: rocketmq.accessKey
            value: "LTAI4Fy1Y6x1KYNz9CLNiso7"
          - name: rocketmq.secretKey
            value: "3FW7D8DYOjvq9xFWvkfCqF97QsPZh8"
          - name: rocketmq.topic
            value: "common"
          - name: rocketmq.groupId
            value: "GID-message"
          - name: rocketmq.tag
            value: "*"
          - name: rocketmq.timeTopic
            value: "time-lapse"
          - name: rocketmq.timeGroupId
            value: "GID-message"
          - name: rocketmq.timeTag
            value: "*"
          - name: rocketmq.name-server
            value:  "mqnamesrv:9876"
          resources:
            limits:
              cpu: "1.0"
              memory: "1Gi"
            requests:
              cpu: "1.0"
              memory: "1Gi"
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime                      # 挂载到容器的目录
      volumes:
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai            # 宿主机的目录
---
# Source: arms-demo/templates/checkoutservice-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkoutservice
  namespace: arms-apm-demo
spec:
  selector:
    matchLabels:
      app: checkoutservice
  template:
    metadata:
      labels:
        app: checkoutservice
    spec:
      containers:
        - name: checkoutservice
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/checkoutservice:v2
          imagePullPolicy: Always
          ports:
          - containerPort: 8089
          env:
          - name: HTTP_PORT
            value: "8089"
          - name: JAEGAR_ENDPOINT
            value: http://tracing-analysis-dc-bj-internal.aliyuncs.com/adapt_eb4zdose6v@9e5d9dc6ff4b44b_eb4zdose6v@53df7ad2afe8301/api/traces
---
# Source: arms-demo/templates/clothservice-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: clothservice
  namespace: arms-apm-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: clothservice
  template:
    metadata:
      labels:
        app: clothservice
      annotations:
        armsPilotAutoEnable: "on"
        armsPilotCreateAppName: "clothservice"
    spec:
      containers:
        - name: clothservice
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/clothservice:v2
          imagePullPolicy: Always
          env:
          - name: spring.cloud.nacos.discovery.server-addr
            value: "nacos-standalone:8848"
          - name: spring.redis.host
            value: "redis"
          - name: spring.redis.password
            value: ""
          - name: spring.datasource.url
            value: "jdbc:mysql://mysql-pod:3306/cart_db?characterEncoding=utf-8&useSSL=false"
          - name: spring.datasource.username
            value: "root"
          - name: spring.datasource.password
            value: "cart@1234"
          - name: rocketmq.nameSrvAddr
            value: "http://1588027496779518.mqrest.cn-shanghai.aliyuncs.com"
          - name: rocketmq.accessKey
            value: "LTAI4Fy1Y6x1KYNz9CLNiso7"
          - name: rocketmq.secretKey
            value: "3FW7D8DYOjvq9xFWvkfCqF97QsPZh8"
          - name: rocketmq.name-server
            value: "mqnamesrv:9876"
          resources:
            limits:
              cpu: "1.0"
              memory: "1Gi"
            requests:
              cpu: "1.0"
              memory: "1Gi"
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime                      # 挂载到容器的目录
      volumes:
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai            # 宿主机的目录
---
# Source: arms-demo/templates/frontend-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: arms-apm-demo
spec:
  selector:
    matchLabels:
      app: frontend
  replicas: 2
  template:
    metadata:
      labels:
        app: frontend
      annotations:
        armsPilotAutoEnable: "on"
        armsPilotCreateAppName: "frontend"
    spec:
      containers:
        - name: frontend
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/frontendservice:v2
          imagePullPolicy: Always
          ports:
          - containerPort: 8080
          env:
          - name: dubbo.registry.address
            value: "nacos://nacos-standalone:8848"
          - name: spring.cloud.nacos.discovery.server-addr
            value: "nacos-standalone:8848"
          - name: spring.redis.host
            value: "redis"
          - name: spring.redis.password
            value: ""
          - name: spring.datasource.url
            value: "jdbc:mysql://mysql-pod:3306/cart_db?characterEncoding=utf-8&useSSL=false"
          - name: spring.datasource.username
            value: "root"
          - name: spring.datasource.password
            value: cart@1234
          - name: rocketmq.nameSrvAddr
            value: "http://1588027496779518.mqrest.cn-shanghai.aliyuncs.com"
          - name: rocketmq.accessKey
            value: "LTAI4Fy1Y6x1KYNz9CLNiso7"
          - name: rocketmq.secretKey
            value: "3FW7D8DYOjvq9xFWvkfCqF97QsPZh8"
          - name: rocketmq.name-server
            value:  "mqnamesrv:9876"
          - name: kafka.bootstrap.servers
            value: "192.168.0.64:9092,192.168.0.65:9092,192.168.0.66:9092"
          - name: kafka.group.id
            value: "testGroupOne"
          - name: kafka.topic
            value: "testTopicOne"
          - name: kafka.enabled
            value: "false"
          - name: web.turn
            value: "true"
          - name: web.pid
            value: "aokcdqn3ly@04f80554356891f"
          resources:
            limits:
              cpu: "1.0"
              memory: "3Gi"
            requests:
              cpu: "0.1"
              memory: "2Gi"
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime                      # 挂载到容器的目录
      volumes:
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai            # 宿主机的目录
---
# Source: arms-demo/templates/mysql.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: "mysql-pod"
  namespace: arms-apm-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "mysql-pod"
  template:
    metadata:
      labels:
        app: "mysql-pod"
    spec:
      containers:
        - name: "mysql-pod"
          image: "registry.cn-shanghai.aliyuncs.com/yxc_test/mysql:v2"
          imagePullPolicy: Always
          ports:
            - containerPort: 3306
              hostPort: 3306
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: cart@1234
          volumeMounts:
            - name: config-volume
              mountPath: /etc/mysql/conf.d/my.cnf
              subPath: my.cnf
            - name: config-sql
              mountPath: /usr/local/work/initsql.sh
              subPath: initsql.sh
          lifecycle:
            postStart:
              exec:
                command: ["bash", "/usr/local/work/initsql.sh"]
            preStop:
              exec:
                command: ["bash", "echo 'nihao' "]
      volumes:
        - name: config-volume
          configMap:
            name: mysql-config
        - name: config-sql
          configMap:
            name: mysql-config
---
# Source: arms-demo/templates/nacos-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nacos-standalone
  namespace: arms-apm-demo
spec:
  selector:
    matchLabels:
      app: nacos-standalone
  template:
    metadata:
      labels:
        app: nacos-standalone
    spec:
      containers:
        - name: nacos-standalone
          image: nacos/nacos-server:latest
          imagePullPolicy: Always
          ports:
          - containerPort: 8848
          env:
          - name: PREFER_HOST_MODE
            value: "hostname"
          - name: MODE
            value: "standalone"
---
# Source: arms-demo/templates/productservice-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: productservice
  namespace: arms-apm-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: productservice
  template:
    metadata:
      labels:
        app: productservice
      annotations:
        armsPilotAutoEnable: "on"
        armsPilotCreateAppName: "productservice"
    spec:
      containers:
        - name: productservice
          image: registry.cn-shanghai.aliyuncs.com/yxc_test/productservice:v2
          imagePullPolicy: Always
          env:
          - name: spring.cloud.nacos.discovery.server-addr
            value: "nacos-standalone:8848"
          - name: spring.redis.host
            value: "redis"
          - name: spring.redis.password
            value: ""
          - name: spring.datasource.url
            value: "jdbc:mysql://mysql-pod:3306/cart_db?characterEncoding=utf-8&useSSL=false"
          - name: spring.datasource.username
            value: "root"
          - name: spring.datasource.password
            value: cart@1234
          - name: rocketmq.nameSrvAddr
            value: "http://1588027496779518.mqrest.cn-shanghai.aliyuncs.com"
          - name: rocketmq.accessKey
            value: "LTAI4Fy1Y6x1KYNz9CLNiso7"
          - name: rocketmq.secretKey
            value: "3FW7D8DYOjvq9xFWvkfCqF97QsPZh8"
          - name: rocketmq.name-server
            value:  "mqnamesrv:9876"
          - name: slowQuery
            valueFrom:
                configMapKeyRef:
                    name: configuration
                    key: SLOW_QUERY
          resources:
            limits:
              cpu: 1.0
              memory: "1Gi"
            requests:
              cpu: 1.0
              memory: "1Gi"
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime                      # 挂载到容器的目录
      volumes:
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai            # 宿主机的目录
---
# Source: arms-demo/templates/redis-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: arms-apm-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis
          resources:
            requests:
              cpu: 100m              #限制cpu的数量为0.1个
              memory: 100Mi      #限制内存为100M
          ports:
            - containerPort: 6379
---
# Source: arms-demo/templates/rocketmq-broker.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mqbroker
  namespace: arms-apm-demo
spec:
  serviceName: mqbroker
  replicas: 1
  selector:
    matchLabels:
      app: mqbroker
  template:
    metadata:
      labels:
        app: mqbroker
    spec:
      containers:
        - name: mqbroker
          image: "registry.cn-shanghai.aliyuncs.com/yxc_test/rocketmq:v1"
          command: ["sh","mqbroker", "-n","mqnamesrv:9876"]
          imagePullPolicy: Always
          ports:
            - containerPort: 10911
            - containerPort: 10911
      imagePullSecrets:
        - name: registry-pull-secret
---
# Source: arms-demo/templates/rocketmq-namesrv.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mqnamesrv
  namespace: arms-apm-demo
spec:
  serviceName: mqnamesrv
  replicas: 1
  selector:
    matchLabels:
      app: mqnamesrv
  template:
    metadata:
      labels:
        app: mqnamesrv
    spec:
      containers:
        - name: mqnamesrv
          image: "registry.cn-shanghai.aliyuncs.com/yxc_test/rocketmq:v1"
          command: ["sh","mqnamesrv"]
          imagePullPolicy: Always
          ports:
            - containerPort: 9876
              protocol: TCP
      imagePullSecrets:
        - name: registry-pull-secret
---
# Source: arms-demo/templates/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  namespace: arms-apm-demo
  annotations:
      kubernetes.io/ingress.class: "nginx"
      nginx.ingress.kubernetes.io/proxy-connect-timeout: "20"
      nginx.ingress.kubernetes.io/proxy-send-timeout: "20"
      nginx.ingress.kubernetes.io/proxy-read-timeout: "20"
      nginx.ingress.kubernetes.io/proxy-body-size: 100m
      nginx.ingress.kubernetes.io/client-body-buffer-size: 100m
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend
                port:
                  number: 8080

码农公寓

背景介绍

前提条件

操作步骤

步骤一 接入Prometheus监控和Kubernetes监控

步骤二 接入demo应用

步骤三 探索集群概览和服务详情

步骤四 查看集群拓扑

总结

附录

相关文章

步骤一接入Prometheus监控和Kubernetes监控

步骤二接入demo应用

步骤三探索集群概览和服务详情

步骤四查看集群拓扑