TCP的连接状态对于我们web服务器来说是至关重要的，尤其是并发量ESTAB；或者是syn_recv值，假如这个值比较大的话我们可以认为是不是受到了攻击，或是是time_wait值比较高的话，我们要考虑看我们内核是否需要调优，太高的time_wait值的话会占用太多端口，要是端口少的话后果不堪设想
一、TCP状态介绍
1、man netstat查看TCP的各种状态信息描述：
LISTEN -侦听来自远方TCP端口的连接请求；
SYN-SENT -在发送连接请求后等待匹配的连接请求；
SYN-RECEIVED -在收到和发送一个连接请求后等待对连接请求的确认；
ESTABLISHED -代表一个打开的连接，数据可以传送给用户；
FIN-WAIT-1 -等待远程TCP的连接中断请求，或先前的连接中断请求的确认；
FIN-WAIT-2 -从远程TCP等待连接中断请求；
CLOSE-WAIT -等待从本地用户发来的连接中断请求；
CLOSING -等待远程TCP对连接中断的确认；
LAST-ACK -等待原来发向远程TCP的连接中断请求的确认；
TIME-WAIT -等待足够的时间以确保远程TCP接收到连接中断请求的确认；
CLOSED -没有任何连接状态；
2、监控原理
[root@Node1 ~]# /bin/netstat -an|awk '/^tcp/{++S[$NF]}END{for(a in S) print a,S[a]}' //通过netstat获取相关值
LISTEN
ESTABLISHED
TIME_WAIT
[root@Node1 ~]#
二、监控脚本编写
1、编写脚本，放于 /etc/zabbix/zabbix_agentd.d/目录下
[root@Node1 zabbix_agentd.d]# cat tcp_status.sh
#!/bin/bash
#This script is used to get tcp and udp connetion status
#tcp status
metric=$
tmp_file=/tmp/tcp_status.txt
/bin/netstat -an|awk '/^tcp/{++S[$NF]}END{for(a in S) print a,S[a]}' > $tmp_file
case $metric in
closed)
output=$(awk '/CLOSED/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
listen)
output=$(awk '/LISTEN/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
synrecv)
output=$(awk '/SYN_RECV/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
synsent)
output=$(awk '/SYN_SENT/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
established)
output=$(awk '/ESTABLISHED/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
timewait)
output=$(awk '/TIME_WAIT/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
closing)
output=$(awk '/CLOSING/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
closewait)
output=$(awk '/CLOSE_WAIT/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
lastack)
output=$(awk '/LAST_ACK/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
finwait1)
output=$(awk '/FIN_WAIT1/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
finwait2)
output=$(awk '/FIN_WAIT2/{print $2}' $tmp_file)
if [ "$output" == "" ];then
echo
else
echo $output
fi
;;
*)
echo -e "\e[033mUsage: sh $0 [closed|closing|closewait|synrecv|synsent|finwait1|finwait2|listen|established|lastack|timewait]\e[0m"
esac
赋予脚本执行权限：
[root@Node1 ~]# chmod a+x /etc/zabbix/zabbix_agentd.d/tcp_status.sh //赋予执行权限
[root@Node1 ~]# ll /etc/zabbix/zabbix_agentd.d/tcp_status.sh
-rwxr-xr-x root root Jan : /etc/zabbix/zabbix_agentd.d/tcp_status.sh
[root@Node1 ~]#
2、添加zabbix配置文件，放于 /etc/zabbix/zabbix_agentd.d/目录下（agent的配置文件 /etc/zabbix/zabbix_agentd.conf 中定义了其他key的包含目录）创建配置文件tcp_status.conf
[root@Node1 ~]# cat /etc/zabbix/zabbix_agentd.d/tcp_status.conf
UserParameter=tcp.status[*],/etc/zabbix/zabbix_agentd.d/tcp_status.sh "$1" //脚本路径
[root@Node1 ~]#
3、确保配置Agent配置文件开启自定义参数UnsafeUserParameters=1
[root@Node1 ~]# grep -n "^[a-Z]" /etc/zabbix/zabbix_agentd.conf
:PidFile=/var/run/zabbix/zabbix_agentd.pid
:LogFile=/var/log/zabbix/zabbix_agentd.log
:LogFileSize=
:DebugLevel=
:Server=172.17.21.208
:ServerActive=172.17.21.208
:Hostname=Node1.contoso.com
:Include=/etc/zabbix/zabbix_agentd.d/*.conf
286:UnsafeUserParameters=1 //1代表允许，0代表关闭
4、重启zabbix-agent服务
[root@Node1 ~]# systemctl restart zabbix-agent.service
备注：因为脚本是把tcp的一些信息存放在/tmp/下，为了zabbix可以读取到我们设置zabbix可以读的权限，确保属主与属组都为zabbix即可
[root@Node1 ~]# chown zabbix.zabbix /tmp/tcp_status.txt //改变属主与属主
[root@Node1 ~]# ll /tmp/tcp_status.txt
-rw-rw-r-- zabbix zabbix Jan : /tmp/tcp_status.txt
5、在zabbix servere服务器上测试,是否能正常获取数据
[root@Node3 ~]# zabbix_get -s 172.17.21.206 -p -k "tcp.status[listen]" [root@Node3 ~]# zabbix_get -s 172.17.21.206 -p -k "tcp.status[timewait]" [root@Node3 ~]# zabbix_get -s 172.17.21.206 -p -k "tcp.status[established]" [root@Node3 ~]#
三、WEB界面配置
1、导入事先准备好的TCP Connection Status模板，Configuration-->Templates-->Import，选择相应的模板，点击Import即可
2、关联模板，Configuration-->Hosts-->Node1--Templates，选择模板,点击Updtae即可
3、查看TCP状态：
Monitoring-->Graphs-->Host(Node1)-->Grahp(TCP Status)
附模板：Template_TCP_Status.xml
<?xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.4</version>
<date>2018-01-18T11:10:22Z</date>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<templates>
<template>
<template>Template TCP Connection Status</template>
<name>Template TCP Connection Status</name>
<description/>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<items>
<item>
<name>CLOSED</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closed]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>CLOSE_WAIT</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closewait]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>CLOSING</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closing]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>ESTABLISHED</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[established]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>FIN_WAIT1</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[finwait1]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>FIN_WAIT2</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[finwait2]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>LAST_ACK</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[lastack]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>LISTEN</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[listen]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>SYN_RECV</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[synrecv]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>SYN_SENT</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[synsent]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>TIME_WAIT</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[timewait]</key>
<delay>60s</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>TCP Status</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
</items>
<discovery_rules/>
<httptests/>
<macros/>
<templates/>
<screens/>
</template>
</templates>
<triggers>
<trigger>
<expression>{Template TCP Connection Status:tcp.status[listen].last()}>500</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>There are too many TCP LISTEN status</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger>
<trigger>
<expression>{Template TCP Connection Status:tcp.status[timewait].last()}>10000</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>There are too many TCP TIME_WAIT status</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger>
</triggers>
<graphs>
<graph>
<name>TCP Status</name>
<width>900</width>
<height>200</height>
<yaxismin>0.0000</yaxismin>
<yaxismax>100.0000</yaxismax>
<show_work_period>1</show_work_period>
<show_triggers>1</show_triggers>
<type>0</type>
<show_legend>1</show_legend>
<show_3d>0</show_3d>
<percent_left>0.0000</percent_left>
<percent_right>0.0000</percent_right>
<ymin_type_1>0</ymin_type_1>
<ymax_type_1>0</ymax_type_1>
<ymin_item_1>0</ymin_item_1>
<ymax_item_1>0</ymax_item_1>
<graph_items>
<graph_item>
<sortorder>0</sortorder>
<drawtype>0</drawtype>
<color>C80000</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[closed]</key>
</item>
</graph_item>
<graph_item>
<sortorder>1</sortorder>
<drawtype>0</drawtype>
<color>00C800</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[closewait]</key>
</item>
</graph_item>
<graph_item>
<sortorder>2</sortorder>
<drawtype>0</drawtype>
<color>0000C8</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[closing]</key>
</item>
</graph_item>
<graph_item>
<sortorder>3</sortorder>
<drawtype>0</drawtype>
<color>C800C8</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[established]</key>
</item>
</graph_item>
<graph_item>
<sortorder>4</sortorder>
<drawtype>0</drawtype>
<color>00C8C8</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[finwait1]</key>
</item>
</graph_item>
<graph_item>
<sortorder>5</sortorder>
<drawtype>0</drawtype>
<color>C8C800</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[finwait2]</key>
</item>
</graph_item>
<graph_item>
<sortorder>6</sortorder>
<drawtype>0</drawtype>
<color>C8C8C8</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[lastack]</key>
</item>
</graph_item>
<graph_item>
<sortorder>7</sortorder>
<drawtype>0</drawtype>
<color>960000</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[listen]</key>
</item>
</graph_item>
<graph_item>
<sortorder>8</sortorder>
<drawtype>0</drawtype>
<color>009600</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[synrecv]</key>
</item>
</graph_item>
<graph_item>
<sortorder>9</sortorder>
<drawtype>0</drawtype>
<color>000096</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[synsent]</key>
</item>
</graph_item>
<graph_item>
<sortorder>10</sortorder>
<drawtype>0</drawtype>
<color>960096</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template TCP Connection Status</host>
<key>tcp.status[timewait]</key>
</item>
</graph_item>
</graph_items>
</graph>
</graphs>
</zabbix_export>