Receiving notifications for UNKNOWN events?

[andrewd]

The manual says the Zabbix cannot perform actions on 'unknown' events, where Zabbix cannot determine the state of a trigger. Why?
If the state cannot be determined then that is certainly a problem because almost certainly something has changed which shouldn't have. I don't understand the logic in not allow actions for these kinds of events.

Anyway I tried a work around, by modifying the an action condition manually in the db. It appeared to work, see screenshot, but a server went down yesterday, spawning unknown events, and my action wasn't triggered. Does anybody have a solution for getting notifications for unknown events?

[richlv]

even if you change values directly in the db, server won't obey incorrect orders

i believe there's a feature request on the tracker to make alerting on unknown states possible - you might want to vote on it

[andrewd]

Thanks, I'll have a look for the item. Meanwhile I'm working on a work around - will post when I'm finished.

[bashman]

I have many "UNKNOWN" events for example in icmpping items.

I can not see "UNKNOWN" events in Monitoring > IT services.

What does "UNKNOWN" in Monitoring > Events mean?.

I can see that in documentation that "UNKNOWN" means that Zabbix cannot evaluate trigger expression, this may happen because of several reasons:
1. server is unreachable
2. trigger expression cannot be evaluated
3. trigger expression has been recently changed

In which case trigger expression cannot be evaluated, let's say for example in "{hostname:icmpping.max(#3)}=0"?.

[morhaven]

Receiving notifications for UNKNOWN events?

ever find a solution on this. I have same issue , I would like to get rid of these Unknows to have valid stats in my reports?

using ICMPPING zabbix 1.8.2

[andrewd]

Coincidentally I just started looking at this again last week. I had stopped trying to find a solution beecause my Zabbix deployment at work had been delayed.

I started looking at this again because at the time i thought it would be hard to solve but while playing around with the zabbix db and I think it's actually quite straightforward. It's about a day's coding in perl, but I'm fitting that around my day job and revising for my VMWare exams. Will post when I've got the script working.

[andrewd]

A solution

Ok, so I've got something working now. I created a script that interrogates the database to find the unsupported items and reports this back to zabbix. For each host you now have 4 items that check the integrity of items and triggers

Triggers in error count - The number of triggers in error
Trigger in error details - A text description of the triggers in error and the items that caused it to be in error.
Items in error count - The number of unsupported items
Items in error details - A list of all the items in error

There are two templates.

One for the zabbix server (Template_unsupported_items_collector) which runs the script, and the 'Template_unsupported_items' should be applied to all hosts. The templates will work for all hosts (SNMP,IPMI,zabbix) because it doesn't talk to the agents just the server.

I've only test this on MySQL but it uses Perl dbi so it should work for other databases.

[The latest version is 0.52 which can be found here]

[bashman]

This query gets the unknown items:

SELECT DISTINCT h.host,h.ip,i.key_ FROM items i, hosts h WHERE i.hostid=h.hostid AND h.status=0 AND i.status=3;

[andrewd]

noticed an error in logic of the trigger check, am working on this now...

[andrewd]

This query gets the unknown items:

SELECT DISTINCT h.host,h.ip,i.key_ FROM items i, hosts h WHERE i.hostid=h.hostid AND h.status=0 AND i.status=3;

Thanks bashman, just clicked refresh and seen your reply.

[andrewd]

The check on the triggers and items is fixed now and updates are correct. Also changed triggers name to display the most recent data instead of the data the first enabled the trigger.

[andrewd]

I imported these templates on a different Zabbix system at work and I discovered that my previous testing didn't properly handle hosts that didn't have Template_unsupported_items applied. I've fixed this.

Note that you may have to change the 'Allowed hosts' for all items in Template_unupported_items to the LAN IP address of your zabbix server. 127.0.0.1 doesn't for all installations.

I've noticed that the very first time you add the template to a host, the 'collector' item (which runs on the server only) may report an error. Not exactly sure why this is but it clears on the next check.

[vpaul]

andrewd, which Zabbix version do you use? I've tried your script and templates with 1.8.4 with no luck. But I really like the approach.

Actually, I'm interested in only UNKNOWN triggers detection for now. So, a dirty fix to the corresponding sql query looks saved me :

# $sql = "select h.host, t.description, i.key_ from items i left join hosts h on i.hostid = h.hostid left join functions f on i.itemid = f.itemid left join triggers t on f.triggerid = t.triggerid where t.status=0 and i.status=3 and h.status=0";

$sql = "select h.host, t.description, i.key_ from items i left join hosts h on i.hostid = h.hostid left join functions f on i.itemid = f.itemid left join triggers t on f.triggerid = t.triggerid where t.status=0 and t.value=2 and h.status=0";

Not sure how much is that correct though... Any feedback would be appreciated.

[andrewd]

I created the templates on Zabbix 1.8.5. What error are you getting?

[vpaul]

Basically I get no any errors. I would say it works but doesn't show any triggers in UNKNOWN state. Not sure about the rest because UNKNOWN is my priority. All counters are just zeros independent of real values.