Server and Proxy stop communicating after 6-12 hours?!

[sai.barker]

Hi,

I'm new to Zabbix and I'm have been racking my brain for the past week trying to get a test lab setup with a Zabbix Server and Proxy on a remote network connected over an L2TP/IPSEC VPN.

I can get everything up and running a few hosts monitored through the proxy and it all seems to work fine... Until it doesn't. It seems random but after 6-12 hours the Zabbix server will stop receiving data from the proxy despite showing online in the frontend. I have re-installed both the server and proxy with various combinations of OS and DB but the problem still keeps happening.

Currently I have the latest versions of Zabbix-Server and Zabbix-Proxy (7.0LTS) running on Ubuntu Server 24.04 and using PostgreSQL. Using this installation guide from Zabbix.

Again, the Zabbix server says the proxy is online through the front end.


There are errors in the Zabbix-Server and Zabbix-Proxy.

Extract from /var/log/zabbix_server.log

979:20240928:180217.750 sending configuration data to proxy "customer01.proxy" at "10.120.0.223", datalen 425, bytes 205 with compression ratio 2.1
982:20240928:180219.255 failed to accept an incoming connection: from 10.120.0.223: reading first byte from connection failed: [11] Resource temporarily unavailable
982:20240928:180225.616 failed to accept an incoming connection: from 10.120.0.223: reading first byte from connection failed: [11] Resource temporarily unavailable
981:20240928:180259.931 sending configuration data to proxy "customer01.proxy" at "10.120.0.223", datalen 425, bytes 205 with compression ratio 2.1
​

Extract from /var/log/zabbix_proxy.log

924:20240928:075946.022 cannot send proxy data to server at "10.240.0.71": empty string received
924:20240928:075952.415 cannot send proxy data to server at "10.240.0.71": empty string received
913:20240928:075955.614 received configuration data from server at "10.240.0.71", datalen 425
924:20240928:075958.814 cannot send proxy data to server at "10.240.0.71": empty string received​

Telnet successfully connects on port 10051 from server > proxy AND proxy > server.
postgresql.service is still running on the server and proxy also.

Any ideas or suggestions of things to try would be greatly appreciated!

[troffasky]

Are you monitoring a zabbix agent on the proxy server from the zabbix server? Does that fail after the same time?
The reason I ask, is that you can troubleshoot a connection to an agent using the zabbix_get tool.

[sai.barker]

That's a good point. No, I don't have a zabbix agent running on the proxy. I'm just relying on ICMP monitoring of the proxy itself and the Proxy status in Administration > Proxies > State.

Is the installation process the same for installing the zabbix agent on the server? just run the same commands on the proxy?

[cyber]

yes... agent installation is same for "everywhere".