Large amount of TIME_WAIT connections

[svenegas]

The problem
------------------
There are a a lot of network conections in TIME_WAIT, the time of pings are very high ( 150 ms ) and snmp monitoring is flaping sometimes.

Zabbix and MySQL Server
-------------------------------------
-) Our Zabbix server 3.4.1 ( Centos 7.3.1611 , 16GRAM, 4 CPU Intel(R) Xeon(R) CPU E5-2658A v3 @ 2.20GHz ) - Linux 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

-) Our Database server ( mysql Ver 14.14 Distrib 5.7.24-27, for Linux (x86_64) using 6.2 ) is working in another host ( 16 GRAM, 16 CPU Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz )

-) Monitoring 2.000 hosts , 150.000 items and 70.000 triggers with zabbix agent in passive mode.


I dont know what is happening .... some hint ?

[kloczek]

Just change kernel settings about net.ipv4.tcp_tw_reuse
https://serverfault.com/questions/23...s-says-netstat
Yes, Linux still has not changed that OOTB like in many other OSes (like Solaris, *BSD and even Win*)
The same issue is not only Zabbix specific but web servers as well and any other when there is may short live connections.

[svenegas]

many thanks !!! Your hint show me the way.

We solved with /proc/sys/net/ipv4/tcp_tw_recycle

And the long time of pings was solved reducing de number of StartPollers ( from 250 to 100 )

[kloczek]

You should start as well thinking about move away from passive to active monitoring if those pollers are used to monitor hosts with zabbix agents

[svenegas]

Thanks , we are testing active monitoring to migrate as soon as posible.