I am monitoring a bunch of servers and had a problem the other day where my zabbix server was not reachable. Since no traffic was coming in zabbix assumed all the monitored servers were offline. Since outbound connection was not working either no alerts could be sent but zabbix will try send them anyways.
Once the zabbix server was reachable again I had a bunch of email alerts all come in all at once and then continue to come in as postfix retried the previous failed emails. Both offline alerts and then the back online alerts. I also send emails to SMS so my phone was going crazy for the next 5 minutes. Is there any way to combine alerts when they all happen at the same time or within a certain time period of each other?
I realize there are things I could do with rules and triggers etc. to make alerting a bit smarter. I will look into that as well. For this post my only question is in regards to combining email alerts.
Once the zabbix server was reachable again I had a bunch of email alerts all come in all at once and then continue to come in as postfix retried the previous failed emails. Both offline alerts and then the back online alerts. I also send emails to SMS so my phone was going crazy for the next 5 minutes. Is there any way to combine alerts when they all happen at the same time or within a certain time period of each other?
I realize there are things I could do with rules and triggers etc. to make alerting a bit smarter. I will look into that as well. For this post my only question is in regards to combining email alerts.