Zabbix connections filling up ip_conntrack table

[froggie]

Hi all,

I've deployed Zabbix at the company I currently work for. So far it served us pretty well, but it seems we're running into problems with our IPtables based firewalls lately.
Our Zabbix server is monitoring about 20 hosts which are on a different subnet, separated by a pair of IPtables firewalls. About two days ago the primary unit started logging the following line:

Jul 13 09:40:14 fw01 kernel: ip_conntrack: table full, dropping packet.

Counting the ip_conntrack table entries it appears monitoring those 20 hosts (each with about 30 items) adds somewhere between 7000 en 9000 entries to this table. Looking at the individual hosts there're 100+ connections in TIME_WAIT state every given moment.
I increased the number of entries in the ip_conntrack tables and after lowering the tcp_fin_timeout on the monitoring host to 30 seconds things seem to improve a bit (about 6000 entries in the table now), but I'm wondering if this is desired behaviour. I was expecting only a couple of connections to each monitored host. What else can I do to keep this undesired overhead under control?

[Alexei]

You may switch to active checks and use agent-side buffering in order to decrease number of established connections. This will also improve performance of Zabbix server due to more efficient processing of active checks.

[froggie]

Thanks for the quick reply. I switched to active checks and it starts to look alot more like it. The number of connections in the ip_conntrack table dropped to around 600 and the load on both the zabbix and db server seems to be reduced a bit. I guess that makes me a happy man.

[eger]

I am noticing this also. But for the local agent on the server. I have 100's of these lines:

tcp        0      0 127.0.0.1:10050             127.0.0.1:44835             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:44834             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:44833             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:45089             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:45088             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:44845             TIME_WAIT
tcp        0      0 127.0.0.1:10050             127.0.0.1:44844             TIME_WAIT

Is there a way to get the agent to close these connections if they are o longer used or reduce the number of connections without using active checking?