Trigger executing falsely even when values do not meet threshold| v3.4

[kapabhi]

Hi ,

I have setup trigger in zabbix with trigger and recovery condition as below:

Trigger Condition:
({SUBMITTED_TASK_COUNT.last(#1)} - {COMPLETED_TASK_COUNT.last(#1)})>10 and ({SUBMITTED_TASK_COUNT.last(#2)} - {COMPLETED_TASK_COUNT.last(#2)})>10

Recovery Condition:
({SUBMITTED_TASK_COUNT.last(#1)} - {COMPLETED_TASK_COUNT.last(#1)})<5 and ({SUBMITTED_TASK_COUNT.last(#2)} - {COMPLETED_TASK_COUNT.last(#2)})<5

Last 2 Values:
SUBMITTED_TASK_COUNT : 128.6 & 143.9
COMPLETED_TASK_COUNT : 128.6 & 143.9

Problem Statement: Trigger and Recovery condition is validated as TRUE at the same time and and it triggers and resolved with 0 second duration. (Both values are fed by a script and recorded via Zabbix trapper and then post-processed as 'change in second')

Any help will be greatly appreciated.

Thanks

[ISiroshtan]

Hey mate.

If both Trigger and Recovery evaluate True - trigger remain in problem state. Recovery expression evaluated only AFTER trigger expression evaluated FALSE.

Most probable cause of issue here is the fact that trigger re-evaluates after ANY of it's condition item is updated. And data for items comes with minimal but still a time difference, causing 2 evaluations of trigger. For example:

Time: 00:00:00
SUBMITTED_TASK_COUNT : 128.6
COMPLETED_TASK_COUNT : 128.6

Time: 00:01:00.000
New data pulled:
SUBMITTED_TASK_COUNT : 143.9
COMPLETED_TASK_COUNT : 128.6
143 - 128 = 15 > 10 - open trigger

Time: 00:01:00.010
New data pulled:
SUBMITTED_TASK_COUNT : 143.9
COMPLETED_TASK_COUNT : 143.9
143-143 = 0 < 10 ---Evaluate recovery--> 0 < 5 --> close the trigger

Effective time trigger was up - 00:00:00.010


As an option to solve it:

If it it's tolerable to delay Alert raising by one extra pull cycle, try adding <time_shift> to last() functions. Like 5 seconds.
This will mean when new data arrives it evaluates values that were 5 seconds back in time, removing time discrepancy introduced by data arriving lag in exchange for delayed Trigger firing.

Alternatively - create a calculated item where perform comparison of SUBMITTED_TASK_COUNT and COMPLETED_TASK_COUNT, and use this delta item in trigger. Calculated items run on it's own set timer so should not be affected by data arrive lag.

Hope it helps mate.