No data after "connection restored"

[donat]

Hi community.

I experience a problem with "no data" after losing and restoring the connection to zabbix agent.

First, I see the error (server log):
"18651:20180406:161932.508 Zabbix agent item "kamailio.core.totalusers" on host "sip.domain.name" failed: first network error, wait for 15 seconds"

Then following row appears (usually in 5-10 seconds after error above):
"18655:20180406:161947.848 resuming Zabbix agent checks on host "sip.domain.name": connection restored"
There are no rows in agent's log when the this issue appears.

I don't ask about, why there are connections loss. I want to know why there is no data after connection is restored.

I have 10+ monitored servers in my network, all of them are monitored with z.agents.
Every day all of them (zabbix agents) are to be restarted (at night) and so that, zabbix server usually notices connections loss and mark hosts as unreachable, after z.agents get back they are marked with "connection restored", but there is no data after that (no data from all of the items, not only my custom user parameters, but others like proc.cpu.util etc).

Of course I can disable restarting of agents, but the same issue appears when the connection is lost by any other reason.

The only way to solve it - is to restart zabbix server (not agents).

I use debug lvl 5 to parse the logs, but cannot see any errors on the server, nor on the agents.
Any hints will be gladly accepted.

[aigars.kadikis]

Hi, donat!

What version of Zabbix server and Zabbix agent do you have?

Please take a look how the zabbix server performs on data gathering proccess:


Also take a look on "Zabbix Server performance" graph.

Regards,
Aigars

[donat]

Zabbix server - 3.0.7
Zabbix agents - 2.2.7

I decided to add a machine which has newer version of agent (3.0.7) and it works fine for it - the data is present for this host.

Took a look at graphs you prompted and I see that there is no performance issues (zabbix server performs on data gathering process).

The problem is that I cannot update version of agents on my most hosts to 3.0.7, cuz the debian version on these machines 8.10
So I got the direction but the way to solve it - is to upgrade OS on hosts, that is too time-consuming for me now.

[donat]

The interesting moment in this issue is - that zabbix_get command executed from server is able to obtain the needed values, but why they are unavailable for items of the web service?
So the agent collects data properly, perhaps there is something wrong in mysql inserts or something like.

Schema as following: zabbix_agent (collects the data) -> sending the data to server -> zabbix_server (gets the data sources) -> trying to insert the data to local mysql storage (failure here?) <- web page is trying to get values from DB (and cannot perform that?).

I want to mention that problem can only be solved with restarting of zabbix server daemon.