目前我们生产环境,每次新加DB机器都要人肉去添加模板,这种方式显然不是一个IT从业人员应该做的。急需完善流程。
看了 grafana官网自带的http api说明也不够清晰,于是自己琢磨了下。有了这篇博客。
1、去grafana里面生成一个api key,用于和grafana进行http交互。
因为很多时候,公司内有很多组织的,处于权限控制考虑,我们的监控模板需要放到对应的组织下,只允许对应的业务人员查看。
这里,我们先新建个组织,例如叫做“会员系统”
然后,去生成1个api key
注意,这个api keys界面只会出现一次,因此需要记住这个key,后续是无法查看到的。
这里,我们记录下关键信息如下:
Authorization: Bearer eyJrIjoiRDd2SXkxU01PUkk4TFVYT0xsM1lQQ1dFaW92dEJIekMiLCJuIjoiZ3JhZmFuYV9hdXRvIiwiaWQiOjE0fQ==
2、添加grafana的数据源,我这里取名为prom25:
3、制作一个标准模板,后面会用到它,我这里简单起见,只用了一个5分钟负载图。(实际生产环境这个模板会展示很多指标,操作方法是一样的)。
这个模板没问题后,我们来导出json格式的模板文件,如下图所示:
我这里导出的json文件全文如下:
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"hideControls": false,
"id": 36,
"links": [],
"rows": [
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prom25",
"decimals": 2,
"fill": 3,
"id": 1,
"legend": {
"alignAsTable": true,
"avg": false,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "node_load5{instance=\"10.0.20.25:9100\"}",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "5分钟负载",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6"
}
],
"schemaVersion": 14,
"style": "dark",
"tags": [],
"templating": {
"list": []
},
"time": {
"from": "now-30m",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "",
"title": "test_dashboard",
"version": 1
}
此外我们可以将这个模板文件导出作为备份,然后删除这个模板。后续会通过http api来添加这个模板。
4、修改上面的这个json文件,修改后的如下:
注意标记红色的第1、2行还有最后3行,都是新加了的内容。
{
"dashboard":
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"hideControls": false,
"id": null,
"links": [],
"rows": [
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prom25",
"decimals": 2,
"fill": 3,
"id": 1,
"legend": {
"alignAsTable": true,
"avg": false,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "node_load5{instance=\"10.0.20.25:9100\"}",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "5分钟负载",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6"
}
],
"schemaVersion": 14,
"style": "dark",
"tags": [],
"templating": {
"list": []
},
"time": {
"from": "now-30m",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "",
"title": "test_dashboard",
"version": 1
}
,
"overwrite": true
}
5、使用chrome的插件restlet_client模拟POST提交操作
如下图,关键点都用红色框起来了。 BODY部分填的就是上面第4步的修改后的json内容。
如果返回值是200OK,则表示post提交成功了。然后去grafana界面查看下吧。没啥问题的话,就可以将这个post请求弄出来了。
点击上图界面的右上角“save as”旁边的小箭头,点击“copy as curl”,即可导出curl格式的命令明细。
我们可看下这个curl的内容,实际上变化的就是3个地方:
1、curl的header部分的认证字段(用来区分属于哪个grafana组织)
2、http的data部分,实际上就是我们的dashboard的全部内容。
3、dashboard的名称。
6、后期
通过上面的几个步骤,我们就可以再努力下,用python搞个web界面,传一个grafana的组织,需要的主机IP即可完成添加。
【grafana里的组织和apikey存在对应关系。 对于http的data部分,实际上只要每次改成一个IP就是一个新的模板(咱DBA的机器,都是一个IP对应一个模板)】