一、拓扑图
二、收集数据
2.1前期准备
创建共享目录,即为了各节点都创建该目录,有两个文件,做数据共享
/home/target/prom-jvm-demo
1、下载文件
jmx_prometheus_javaagent-0.3.1.jar
2、创建文件
[root@manager prom-jvm-demo]# cat simple-config.yml
---
rules:
- pattern: ".*"
2.2启动容器
注:6060作为java监控数据暴露端口,每一个服务单独占用一个端口,即启动java服务便伴随着启动java监控服务
2.2.1 docker 启动(测试)
docker run -d \
--name tomcat-1 \
-v /home/target/prom-jvm-demo:/jmx-exporter \
-e CATALINA_OPTS="-Xms64m -Xmx128m -javaagent:/jmx-exporter/jmx_prometheus_javaagent-0.3.1.jar=6060:/jmx-exporter/simple-config.yml" \
-p 6060:6060 \
-p 8012:8080 \
10.0.0.194:4999/tomcat:8.5.61
2.2.2 docker 启动
docker service create --name bj_ywjy_yxsmgr2 \
--publish 6061:6060 \
--publish 8081:8080 \
--mount type=bind,src=/home/target/bj_ywjy_yxsmgr,dst=/usr/local/tomcat/webapps \
--mount type=bind,src=/home/target/bj_ywjy_yxsmgr/logs,dst=/usr/local/tomcat/logs \
--mount type=bind,src=/home/target/prom-jvm-demo,dst=/jmx-exporter \
--env CATALINA_OPTS="-Xms64m -Xmx128m -javaagent:/jmx-exporter/jmx_prometheus_javaagent-0.3.1.jar=6060:/jmx-exporter/simple-config.yml" \
--network springcloud-overlay \
--log-driver=gelf --log-opt gelf-address=udp://10.0.0.184:12201 \
--log-opt tag=bj_ywjy_yxsmgr2 10.0.0.194:4999/tomcat:8.5.61
2.2.3 访问获取到的数据
http://10.0.0.196:6060/metrics
三、prometheus配置
3.1 将获取到的java数据关联到prometheus中
[root@autodeploy prometheus-2.25.2]# cat prometheus.yml
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus' # metrics_path defaults to '/metrics'
# scheme defaults to 'http'. static_configs:
- targets: ['10.0.0.189:9090'] - job_name: 'Container monitoring'
static_configs:
- targets:
- "10.0.0.194:8888"
- "10.0.0.195:8888"
- "10.0.0.196:8888" - job_name: 'bj_ywjy_yxsmgr'
static_configs:
- targets:
- "10.0.0.194:6060"
- job_name: 'bj_ywjy_yxsmgr2'
static_configs:
- targets:
- "10.0.0.194:6061"
配置jvm监控规则可根据 cadvisor+prometheus+alertmanager+grafana完成容器化监控告警(一)配置参数调整改进,这里就不体现了!
3.2重启prometheus
[root@autodeploy prometheus-2.25.2]# cat start.sh
#!/bin/bash
pid=`ps -ef|grep prometheus|grep -v grep|awk '{print $2}'`
kill -9 $pid
nohup ./prometheus --config.file=prometheus.yml &
3.3访问测试
http://10.0.0.189:9090/
四、alertmanager配置告警
4.1告警模板
[root@autodeploy rules]# cat weixin.tmpl
{{ define "wechat.default.message" }}
{{ range $i, $alert :=.Alerts }}
========监控报警====
告警状态: {{ .Status }}
告警级别: {{ $alert.Labels.severity }}
告警类型: {{ $alert.Labels.alertname }}
告警应用: {{ $alert.Labels.name }} {{ $alert.Labels.job }}
告警主机: {{ $alert.Labels.instance }}
当前最新值: {{ $alert.Annotations.summary }}
告警详情: {{ $alert.Annotations.description }}
告警时间: {{ (.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}
========end=========
{{ end }}
{{ end }}
4.2重启alertmanager
[root@autodeploy alertmanager-0.21.0]# cat start.sh
#!/bin/bash
pid=`ps -ef|grep aler|grep -v grep|awk '{print $2}'`
kill -9 $pid
nohup ./alertmanager --config.file=alertmanager.yml --storage.path=data --log.level=debug &
五、微信告警展示