I know this is going to sound stupid, but bear with me for a minute here.
We have 2 active proxys in a proxy group collecting data, in hybrid mode, configured to store data for 24 hours if the zabbix server is offline.
We had an unexpected outage in the middle of the night on the MySQL backend database. Zabbix server could not connect to the database for a 3hr period until I logged in remotely and fixed it.
Zabbix Server cannot possibly write the data to MySQL, but still appear to accept data from proxys!?! That data goes where? I don't think it goes anywhere. It just vanishes.
Zabbix Server was sending emails advising the database was down, so it's not like it thought everything was OK.
WHY would Zabbix Server continue to accept data from proxys in this scenario?
Zabbix Server should stop accept data, causing proxys to hold it offline until Zabbix Server is ready and ABLE to receive the data again.
Is there a way to configure Zabbix Server to NOT accept data if it can't write it anywhere?
If not... I am considering a script on Zabbix Server that checks connectivity and read/write access to the database. If the checks fail the script could either A) Stop the Zabbix Server service, or B) Prevent proxy connections via iptables rules...
I suspect that option B could wreak havoc on the proxy load balancing algorithm, but haven't done any testing yet.
We have 2 active proxys in a proxy group collecting data, in hybrid mode, configured to store data for 24 hours if the zabbix server is offline.
We had an unexpected outage in the middle of the night on the MySQL backend database. Zabbix server could not connect to the database for a 3hr period until I logged in remotely and fixed it.
Zabbix Server cannot possibly write the data to MySQL, but still appear to accept data from proxys!?! That data goes where? I don't think it goes anywhere. It just vanishes.
Zabbix Server was sending emails advising the database was down, so it's not like it thought everything was OK.
WHY would Zabbix Server continue to accept data from proxys in this scenario?
Zabbix Server should stop accept data, causing proxys to hold it offline until Zabbix Server is ready and ABLE to receive the data again.
Is there a way to configure Zabbix Server to NOT accept data if it can't write it anywhere?
If not... I am considering a script on Zabbix Server that checks connectivity and read/write access to the database. If the checks fail the script could either A) Stop the Zabbix Server service, or B) Prevent proxy connections via iptables rules...
I suspect that option B could wreak havoc on the proxy load balancing algorithm, but haven't done any testing yet.
Comment