We are running Zabbix 1.8.11. Two days ago we had a major power outage that affected many systems and caused a lot of triggers to fire. The outage was resolved two days ago, but I am still receiving e-mail alerts about affected systems going down and coming up. The systems indicated by these messages do not currently have any problems, as seen in Dashboard. Looking at headers of these e-mails I can see that they haven't been held in some mail queue but are freshly generated by Zabbix server. I've looked at Administration > Queue, but there is nothing that has been queued for more than 1 minute. I've also checked the database server (separate PostgreSQL server) but the load there is normal and there are not any long-running database client processes. I have restarted the Zabbix server and PostgreSQL server (daemons, not operating systems) but this didn't solve the problem. Zabbix server log shows nothing unusual.
Where else should I look in order to further troubleshoot the problem? Or if I can't find the root cause, can I somehow tell Zabbix to forget about triggers that were fired more than 24h ago?
Where else should I look in order to further troubleshoot the problem? Or if I can't find the root cause, can I somehow tell Zabbix to forget about triggers that were fired more than 24h ago?
Comment