Kubernetes & Docker监控之最后一章,前三篇如下:
2、 kubernetes+docker监控之Docker监控——cadvisor
3、 Docker监控——Cadvisor+InfluxDB+Grafana搭建过程
1)、heapster下载和启动:
(1)、heapster镜像下载:
docker pull index.tenxcloud.com/google_containers/heapster:v1.1.0 |
(2)、heapster单独作为容器启动:
docker run -d index.tenxcloud.com/google_containers/heapster:v1.1.0 "--source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth=" "--sink=influxdb:http://192.168.16.234:8086" |
参数说明:
--source :指定kube-apiserver,这里使用https,需要注意下
--sink :指定influxdb,heapster自动创建数据库【k8s】
inClusterConfig :Use kube config in service accounts associated with heapster's
namesapce. (default: true)
kubeletPort :kubelet port to use (default: 10255)
kubeletHttps : whether to use https to connect to kubelets (default: false)
apiVersion : API version to use to talk to Kubernetes. Defaults to the version
in kubeConfig.
insecure :whether to trust kubernetes certificates (default: false)
auth :client auth file to use. Set auth if the service accounts are not usable.
useServiceAccount :whether to use the service account token if one is mounted
at /var/run/secrets/kubernetes.io/serviceaccount/token (default: false)
heapster容器,启动过程:
[root@localhost kube-1.2]# docker run -d index.tenxcloud.com/google_containers/heapster:v1.1.0 "--source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth=" "--sink=influxdb:http://192.168.16.234:8086"
b86aaaaa88ccf648120611254bfeb0078a8b9dd853d4b5a2d6a0abb97743bf1d
[root@localhost kube-1.2]# docker logs b86aaaaa8
I0714 07:05:18.116787 1 heapster.go:65] /heapster --source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth= --sink=influxdb:http://192.168.16.234:8086
I0714 07:05:18.117158 1 heapster.go:66] Heapster version 1.1.0
I0714 07:05:18.239220 1 configs.go:60] Using Kubernetes client with master "http://192.168.16.100:8080" and version "v1"
I0714 07:05:18.239270 1 configs.go:61] Using kubelet port 10250
I0714 07:05:31.255132 1 influxdb.go:223] created influxdb sink with options: host:192.168.16.234:8086 user:root db:k8s
I0714 07:05:31.255170 1 heapster.go:92] Starting with InfluxDB Sink
I0714 07:05:31.255177 1 heapster.go:92] Starting with Metric Sink
I0714 07:05:31.271159 1 heapster.go:171] Starting heapster on port 8082
I0714 07:06:05.000403 1 manager.go:79] Scraping metrics start: 2016-07-14 07:05:00 +0000 UTC, end: 2016-07-14 07:06:00 +0000 UTC
E0714 07:06:05.000504 1 kubelet.go:270] No nodes received from APIserver.
I0714 07:06:05.000520 1 manager.go:152] ScrapeMetrics: time: 5.756µs size: 0
I0714 07:06:05.365486 1 influxdb.go:201] Created database "k8s" on influxDB server at "192.168.16.234:8086"
[root@localhost kube-1.2]#
[root@localhost kube-1.2]# docker ps |grep heapster
b86aaaaa88cc index.tenxcloud.com/google_containers/heapster:v1.1.0 "/heapster --source=k" 2 minutes ago Up 2 minutes furious_mahavira
[root@localhost kube-1.2]#
看到influxdb已经有k8s的数据库和表了,剩下的就是grafana搜索和显示了
[root@localhost kube-1.2]# influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.9.6
InfluxDB shell 0.9.6
>
>
> show databases
name: databases
---------------
name
_internal
cadvisor
k8s
>
> use k8s
Using database k8s
>
> SHOW MEASUREMENTS
name: measurements
------------------
name
cpu/node_reservation
cpu/node_utilization
cpu/usage
cpu/usage_rate
filesystem/limit
filesystem/usage
memory/major_page_faults
memory/major_page_faults_rate
memory/node_reservation
memory/node_utilization
memory/page_faults
memory/page_faults_rate
memory/usage
memory/working_set
network/rx
network/rx_errors
network/rx_errors_rate
network/rx_rate
network/tx
network/tx_errors
network/tx_errors_rate
network/tx_rate
uptime
2)、InfluxDB安装与配置:
请参考之前Cadvisor+InfluxDB+Grafana的内容,http://my.oschina.net/fufangchun/blog/718382
3)、Grafana安装与配置:
(1)、Grafana安装:
请参考之前Cadvisor+InfluxDB+Grafana的内容,http://my.oschina.net/fufangchun/blog/718382
(2)、Grafana配置:
Ⅰ、配置influxdb数据源:
Ⅱ、新建展示页:
展示页,用来区分不同的influxdb数据源
Ⅲ、新建数据项:
根据特定的条件(sql),从influxdb查询相关的数据,并展示。
Grafana会自动显示k8s库的字段,可以根据需求进行选择,每个where后面,加上:【pod_name =~ /^\.*/】,按照pod名称分类
SELECT last(value) FROM k8s."default"."network/tx" WHERE pod_name =~ /^\.*/ AND time > now() - 30m GROUP BY time(10s), pod_name |
Ⅳ、完整的grafana展示:
这里上传可能看不清,可以去百度盘下载,我这里是有些数据是分别做了,按照pod和namespace进行分组的,有多租户的情况,比较实用,可以根据情况,自行定义。