Hello,
I'm running Zabbix 7.2 and I'm trying to create a specific alert. I'm so close, but I'm stuck on a duplicate alert "race condition" problem.
My Goal: Send a single email alert when an external consultant logs in via RDP to one of our Windows Server 2016 machines. I want to ignore logins from our internal admin accounts.
What I Have Done (My Setup):
What's Working: The preprocessing is working perfectly. It filters out non-RDP logins, and it correctly extracts the username (e.g., my_admin_user).
What's NOT Working (The Problem): When I RDP to a server, a single login generates two alerts at the exact same timestamp.
My "Problems" dashboard shows this (I am logging in as my_admin_user):
Time Problem
03:30:15 PM External RDP Logon on SERVER-A: my_admin_user
03:30:15 PM External RDP Logon on SERVER-A: my_admin_user
My preprocessing is correctly extracting the username for both events, and my preprocessing step 3 (checking for empty string) is not catching a "ghost" event. It seems my server is just sending two identical, valid events, and Zabbix is processing both.
What I Have Tried (That FAILED):
I'm in a classic race condition, and the events are processed too fast for Zabbix to catch the duplicate. Here's what I've tried:
Question: How can I reliably suppress this second, duplicate alert?
My final goal is to add a 4th preprocessing step to exclude users (like my_admin_user), but I can't do that until I solve this duplicate alert problem. Is there a better preprocessing step, trigger function, or correlation method to handle two events that arrive at the same identical timestamp?
Thank you!
I'm running Zabbix 7.2 and I'm trying to create a specific alert. I'm so close, but I'm stuck on a duplicate alert "race condition" problem.
My Goal: Send a single email alert when an external consultant logs in via RDP to one of our Windows Server 2016 machines. I want to ignore logins from our internal admin accounts.
What I Have Done (My Setup):
- Windows Server: "Audit Logon Events" (Success) is enabled. I can see Event ID 4624 in the Security log for every RDP login.
- Zabbix Item: I have an active agent item on my Windows template.
- Name: RDP Logon Detection
- Key: eventlog[Security,,"Success",,"4624",,skip]
- Type: Log
- Preprocessing (3 Steps): This is where the main logic is.
- Step 1: Matches regular expression | Logon Type:[\s\t]+10
- On fail: Discard value
- Step 2: Regular expression | New Logon:[\s\S]*?Account Name:[\s\t]+(.*?)\r?\n
- On fail: Discard value | Output: \1
- Step 3: Does not match regular expression | ^$
- On fail: Discard value
- Step 1: Matches regular expression | Logon Type:[\s\t]+10
- Zabbix Trigger:
- Expression: length(last(/.../eventlog[Security,,"Success",,"4624",,skip]))>0
- PROBLEM event generation mode: Multiple
- Tags: I added a tag rdp_user with the value {ITEM.LASTVALUE}.
What's Working: The preprocessing is working perfectly. It filters out non-RDP logins, and it correctly extracts the username (e.g., my_admin_user).
What's NOT Working (The Problem): When I RDP to a server, a single login generates two alerts at the exact same timestamp.
My "Problems" dashboard shows this (I am logging in as my_admin_user):
Time Problem
03:30:15 PM External RDP Logon on SERVER-A: my_admin_user
03:30:15 PM External RDP Logon on SERVER-A: my_admin_user
My preprocessing is correctly extracting the username for both events, and my preprocessing step 3 (checking for empty string) is not catching a "ghost" event. It seems my server is just sending two identical, valid events, and Zabbix is processing both.
What I Have Tried (That FAILED):
I'm in a classic race condition, and the events are processed too fast for Zabbix to catch the duplicate. Here's what I've tried:
- Trigger Logic (Debouncing):
- ... and count(...,5s)<2 (also tried 1s): Still got two alerts.
- ... and count(...,1s)=1: This was worse; it caused the alert to flap (created a Problem and immediately Resolved it).
- Event Correlation (in Zabbix 7.2):
- I created a global correlation rule (Data collection > Event correlation) to Close new event.
- I tried matching by a static tag (class=...) and also by my dynamic tag (Event tag pair where Old event tag: rdp_user and New event tag: rdp_user).
- Both correlation rules failed. It seems the Zabbix server processes both events before it can register the first one as an "old problem" to correlate against.
Question: How can I reliably suppress this second, duplicate alert?
My final goal is to add a 4th preprocessing step to exclude users (like my_admin_user), but I can't do that until I solve this duplicate alert problem. Is there a better preprocessing step, trigger function, or correlation method to handle two events that arrive at the same identical timestamp?
Thank you!
Comment