Trigger firing when I don't think it should?

[cdecarlo]

Interesting...I added new items/triggers Friday evening that returned integer values (0/1 instead of ERROR/OK respectively) for these HTTP URL monitors. Over the weekend the trigger that is based on the count function of the character data return a few dozen times and the triggers based on the integer values did not fire at all. The character based trigger is using the count() function while the integer based values are using the max() function. If I wrote them correctly they should have the same exact triggering logic. So it looks like something strange is going on with the count() function when using character data.

Here are my trigger expressions:

{zabbix-proxy-20.sys.courtnet.org:vm-esig-appp1_HSM_Pool_Health.count(#3,"ERROR",like)}>1

(This one fired dozens of times over the weekend)

{zabbix-proxy-20.sys.courtnet.org:vm-esig-appp1_HSM_Pool_Health-integer.max(#3)}=0

(This one never fired over the weekend)

[cdecarlo]

O.K. I guess I spoke too soon...so I deleted the items that returned character data and now only have the items returning integer values and their triggers. These are firing again with the max(#3) = 0 function now...so it looks like there is definitely an issue with functions taking the number of collected values as a parameter. I'm going to try changing it to the number of seconds for three collections + 30 seconds to see if that makes a difference.

[cdecarlo]

Changing to a time interval instead of a number of collections did not make any difference. I've submitted a JIRA ticket to Zabbix (https://support.zabbix.com/browse/ZBX-17646)

[cdecarlo]

OMG!!

I am such a complete idiot!! I totally forgot that my test Zabbix server was still up and running. We have been working from home with the COVID-19 pandemic for the last several weeks (with a few weeks of not working right when it started), and I totally forgot that I had a test Zabbix server running on one of my test boxes. I was writing an application using the Zabbix API to dump our monitoring inventory from our existing Hyperic monitoring server (which has been end-of-life). I was testing the import of all the items/triggers/actions on a test server and I left it running when we were all sent home. Since I've been working from home I've been setting up the actual production Zabbix servers. That's were the other alerts were coming from (I never modified those triggering functions on the test server).

Changing the email template is what finally clued me in...I was getting some emails for items with the new template and some with the original template. The only alerts that I received for these items was using the original email template and were coming from the test server.

I apologize for the wild goose chase!