Triggers can break Zabbix!

[DSon]

Hi,

I've been working with Zabbix since 1.6.4 and have noticed that it is too easy to break.

My concern is that if a trigger cannot be evaluated (for whatever reason), not only do other triggers fail to evaluate, but data collection stops too!

This is more noticeable since I transitioned to a DM setup (using proxies).

If a proxy hasn't forwarded data to the server for more than a few minutes, ANY trigger using the time-based operands (e.g. MIN, MAX) fails to work. Not only this, but all other triggers fail, and data collection from all proxies halts.

The only way to cure the problem is to disabled ALL triggers (most of which I admit are using MIN/MAX), and then reboot ALL proxies.

Does anyone here have any solution (other than not using the MIN/MAX operands)?

I would use trigger dependencies to try and prevent this from happening however this functionality is broke in 1.8.3 too (to be fair, I can see it has been fixed in the latest release candidate).

Also, I don't believe setting a dependency is really the answer. I understand that if a proxy hasn't sent any data for a while, the trigger logic won't work because it is time-sensitive. However, a "faulty" trigger shouldn't bring down an entire distributed Zabbix setup - surely?

Is there no way the Zabbix team can configure a trigger to disable itself if it can't evaluate the data?

Along with poor event log problem tracking (I've posted about this in other threads), this is becoming a major turn-off for me. Which is a shame because I really appreciate how far Zabbix has come in other ways - it's mostly a great product.

Look forward to hearing other people's thoughts on this.

thanks
Danny.

[qix]

Hello,

I've been away from the forums for a while (since zabbix 1.4.4) and have just started playing with zabbix 1.8.3 and zabbix proxies, which is a new thing for me. It seems very promising.

Do I understand you correctly that when a proxy doesn't send any data (for a while) to the zabbix server, it just freezes the whole trigger evaluation process?

If I understand you correctly, this would mean that a faulty uplink to a remote location could bring an entire Zabbix installment to it's knees.

Could you please verify that I have understood you correctly or explain it further if I misinterpreted your post?

[DSon]

Hi there,

Yes I agree, the proxy solution is very promising however my concern is that Zabbix - in general - doesn't handle trigger evaluation very well.

In particular, when using the functions MIN/MAX (and others that support time-based parameters), Zabbix breaks and cannot process new data.

This is hilighted with a proxy setup because if the data sent to the server is behind, trigger logic that uses time (e.g. values based on a number of seconds worth of data) not only fails to evaluate, but more worryingly this failure prevents further collection of data.

Here is an example of a trigger which causes these problems:

{Template_FTP:service_state[MSFtpsvc].min(300)}#0 | {Template_FTP:service_state[MSFtpsvc].nodata(300)}#1

Note: the above trigger SHOULD manage to survive no data being received

Maybe I'm doing something wrong somewhere?

I have searched the forums for similar problems and the only reply so far has been to use a trigger dependency - i.e. on the proxy heartbeat. This doesn't solve the problem of triggers stopping Zabbix from processing other data however.

I would like to know if others have experienced this, and if the Zabbix team can offer advice on this problem.

I suspect the answer is that triggers need to "auto disable" after so many evaluation failures - and then possibly re-enable after a certain period of time (or can be re-enabled by the administrator).

Danny.

[DSon]

Bump!

Any thoughts, anyone?

DSon.

[fmrapid]

Hello,

Have you opened a ticket with the Zabbix team to notify them of this issue.

This is a very serious concern.

fmrapid

[DSon]

Bug created..

https://support.zabbix.com/browse/ZBX-3386