I have encountered a consistent issue regarding Trigger Actions evaluation in Zabbix, specifically during the Recovery (Resolved) phase when a specifically tagged action overlaps with a broader action that covers the entire host.
Setup Details:
The Issue:
Expected Behavior: Zabbix should evaluate recovery events with the exact same strict condition matching as problem events, ensuring that specific actions (like those using unique event tags and dedicated media types) independently process and send their own resolved notifications without being overridden, intercepted, or hijacked by broader host-wide actions.
Logically, any Problem messages sent to a specific channel must have their corresponding Problem Resolved messages delivered to that exact same channel.
Any insights or fixes for this behavior would be greatly appreciated.
Setup Details:
- I have two distinct Trigger Actions configured:
- Action A (Specific / SRE Team): Contains specific conditions and a dedicated event tag (e.g., sre_call: yes), targeting a dedicated Mattermost media type/channel assigned specifically to the SRE team.
- Action B (Broad / Network Team): Covers the entire host (no specific tags, targeting all triggers for the host), targeting the Network team's Mattermost media type/channel.
- Both Problem and Resolved operations are defined for both actions.
- Troubleshooting Steps Taken: To thoroughly verify this, a separate Mattermost media type was created, and the word "TEST" was explicitly added to the Problem and Resolved message templates of the SRE action. Furthermore, this dedicated media type was registered in both the Trigger Action operations and the specific SRE User profile.
The Issue:
- When a Problem occurs, Zabbix evaluates the conditions correctly. Both Action A and Action B trigger their respective notifications, and both groups receive the Problem alert successfully (with the "TEST" keyword appearing correctly in the SRE channel).
- However, when the trigger state changes to Resolved, Zabbix's evaluation engine exhibits a conflict during Action Execution/Overlap. Because Action B covers the entire host without strict tags, Zabbix's evaluation engine fails to properly isolate the specific tagged action (Action A) during recovery.
- As a result, the resolved event notification for Action A is hijacked/overwritten or incorrectly redirected and sent via Action B to the Network team's Mattermost channel, while the Action Log falsely indicates that the message was successfully dispatched for Action A.
- If I temporarily disable Action B (the broad host-wide action), the Resolved notification successfully and correctly reaches the SRE team's Mattermost channel via Action A. This confirms that an overlap conflict in action evaluation occurs exclusively during the recovery phase when a host-wide broad action exists alongside a specifically tagged action.
Expected Behavior: Zabbix should evaluate recovery events with the exact same strict condition matching as problem events, ensuring that specific actions (like those using unique event tags and dedicated media types) independently process and send their own resolved notifications without being overridden, intercepted, or hijacked by broader host-wide actions.
Logically, any Problem messages sent to a specific channel must have their corresponding Problem Resolved messages delivered to that exact same channel.
Any insights or fixes for this behavior would be greatly appreciated.
Comment