"Active agent" triggering several times a day and then resolving under 2 mins

[Aat_Lab32]

Hi all,

I have an issue where multiple hosts (about 50, sometimes more), both Windows and Linux are triggering the "Active agent is not available" all at the same time, two or three times a day (and raising respective incidents) and then resolving within 2-3 mins. These are across different proxies.

Initially I was getting an increasing number (once a day) of events for "Failed to fetch stats data cache data %". I noticed that the time difference on the server and proxies was out by a few min. I fixed the issue and now the agent active triggers are going crazy. There are no items of note on any logs (server log and even host logs).

Zabbix version 7.0.13 and only Agent 2 is used.

[DuaerTeam]

Aat_Lab32, since around 50 hosts behind different proxies flip at the same moment and recover within a couple of minutes, I'd look at the server side before the agents. Active agent availability depends on the heartbeats the agents send, and those heartbeats reach the server through the proxies. A short stall anywhere on that path makes every host look unavailable at once.

Things worth checking around the exact event times:
1. The server's internal health graphs: busy percentage of the history syncer, preprocessing and trapper processes, the write cache, and the queue. A spike lining up with the alerts would explain it.
2. Each proxy's last-seen time and data delay as seen by the server, to tell whether all proxies paused together.
3. Time sync. Since the clocks were out by minutes before, make sure server, proxies and database all use NTP rather than a one-off correction, and that they haven't drifted again.
4. Whether anything scheduled runs then: backups, housekeeping, database maintenance.

Do the events always happen at roughly the same times of day?