Problem using regexp or other way to monitor an eventlog value

[kehall]

Hi

I want to trigger an alert when a value inside an eventlog entry reaches a threshold... There is no other way to get this data other than eventlogs.

e.g. the event text contains

"The table xyz has 7000 MB of free space left in after compacting"

I want to trigger an alert if the value drops below a value (e.g. 5000).

I have tried

regexp(The table .* has ([1-4][0-9][0-9][0-9]|[0-9][0-9][0-9]|[0-9][0-9]|[0-9]) MB of free space)

but the front end won't allow that (doesn't like the brackets)


and both:

regexp(The table .* has [1-4][0-9][0-9][0-9]|[0-9][0-9][0-9]|[0-9][0-9]|[0-9] MB of free space)

regexp(The table .* has \([1-4][0-9][0-9][0-9]|[0-9][0-9][0-9]|[0-9][0-9]|[0-9]\) MB of free space)

these end up triggering on any event that has numbers in there - ignoring the text.

How do I match the exact text and numbers (from 0-5000), or even better, use capture groups or backreferences so I can do something like regexp(The table .* has (\d{0,5}) MB of free space) < 5000 in the trigger

?

Regards,

Keith.

[zalex_ua]

"The table xyz has 7000 MB of free space left in after compacting"
I want to trigger an alert if the value drops below a value (e.g. 5000).

your regexp is generally fine, but one part of it can be more simple:

([1-4]?[0-9]{0,3})

Read this is very nice article.

and as result you trigger should be like:
{HostName:eventlog[Application].regexp(The table .* has ([1-4]?[0-9]{0,3}) MB of free space)}#1

note the characters "#". it is very important in you case.

Why you do not post the full trigger expression?



p.s. Eventlog monitoring - is a special art, which I knew deep.

[kehall]

and as result you trigger should be like:
{HostName:eventlog[Application].regexp(The table .* has ([1-4]?[0-9]{0,3}) MB of free space)}#1
note the characters "#". it is very important in you case.

Thanks - I should have realised the optimal regex but in a way was trying to explain the issue with parentheses!

Also I'm not sure on the #1, surely #0 to match the regex (this is what is built from Items->Create Log Trigger)? #1 seems to trigger all the time on any event :/

When I try to enter the regex above into the Items->Create Log Trigger I get

* Unnecessary symbols detected: Check expression part starting from ' MB of free space)})#0) '
* Trigger [test ]: cannot update

When I take out the parentheses around "[1-4]?[0-9]{0,3}", then it accepts it, but it seems to trigger on any event item that contains numbers (e.g. "Connections: accepted: 192.168.2.10::60949" triggers it!), and yet it ignores the requirement for the text either side to match "The table .* has" and "MB of free space" :/

the trigger as it currently stands is

(({HostName:eventlog[Application].regexp(The table .*Store.* has ([1-4]?[0-9]{0,3}) MB of free space)})#0)

As I only need to report on tables with the name Store in them.

Right now it seems to be not triggering on everything

How do we use parentheses in regex in Zabbix? e.g. if to match a mid-string value from 0-255 and the regex for that was \b([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\b, I believe parentheses are required in that case.

I tried doing some testing on events I can generate easily (as there doesn't seem to be any easy way to test triggers otherwise)

so, for VNC where events are like "Connections: accepted: 192.168.2.10::60949"

I created a regex trigger of Connections: accepted: .*::[0-9]{0-5}
which in turn created a trigger expression of

(({Hostname:eventlog[Application].regexp(Connections: accepted: .*::[0-9]{0-5})})#0)

yet this doesn't seem to fire at all, despite entries in Latest data history

[2010.Nov.28 22:11:27] 2010.Nov.28 22:11:07 WinVNC4 Information 1 Connections: accepted: 192.168.2.10::63634

I think I must be missing something straightforward here!

Regards,

Keith.

[kehall]

Does anyone have any examples of fairly complex regexp for event monitoring? e.g. use of brackets, etc - as I cannot get this working at the moment

[zalex_ua]

I'm working right now on writing article for a comprehensive Eventlog monitoring.
I hope in a few weeks article will be ready.
Some examples you can find in documentation.

[kehall]

ok I look forward to that, but in the meantime, any ideas for how to input the following regex which requires parentheses:

\b([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\b

cannot input that as it contains "(" and ")" and the front end fails with an error "Unnecessary symbols detected: Check expression part starting from"

then more crucially right now, I need help why these expressions are failing:

(({Hostname:eventlog[Application].regexp(Connections: accepted: .*::[0-9]{0,5})})#0)

fails to trigger when Latest data shows

Timestamp Local time Source Severity Event ID Value
[2010.Dec.09 20:52:05] 2010.Dec.09 20:52:03 WinVNC4 Information 1 Connections: accepted: 192.168.2.10::52388

I tried

regexp(Connections: accepted: 192\.168\.2\.10::[0-9]{0,5})

that also fails. I can only assume it doesn't like the [0-9]{0,5} which is valid regex :/

if I use regexp(Connections: accepted: 192\.168\.2\.10::.*) it works...

I don't have any other easily generatable events that I can use to test the regex for the following

regexp(The table .*Store.* has [1-4]?[0-9]{0,3} MB of free space)

which is failing to trigger for event text

The table "OrderHistory Store" has 278 MB of free space

I wonder if it doesn't like the {} braces :/

Help !

[zalex_ua]

Try to declare you expression at global level and after use it in trigger.

[kehall]

That works!

I can only assume regex in the trigger itself is fairly useless unless this is a bug?

Regards,

Keith.

[kehall]

hmmm....

I'm getting triggering on events which do not match the regex...

the text which caused the trigger:

eseutil (25464) The database engine (8.03.0083.0000) started a new instance (0).

For more information, click http://www.microsoft.com/contentredirect.asp.


The trigger expression is:

(({Template_Win2003_Exch_2007:eventlog[Application].regexp(@evtregex122)})#0)
Event generation: Normal

The global regex 'evtregex122' is:

The table .*Store.* has [1-4]?[0-9]{0,3} MB of free space
Expected result: Result is TRUE
Case sensitive: No

If I put in the test string
The table "OrderHistory Store" has 278 MB of free space
and click Test, the Result is TRUE

If I put the above string
eseutil (25464) The database engine (8.03.0083.0000) started a new instance (0).

For more information, click http://www.microsoft.com/contentredirect.asp.

and click Test, the Result is FALSE

so why should the trigger be occurring?

Keith

[kehall]

Still having reliability issues with EventLog regex monitoring, triggers are firing fairly consistently but not reliably, and importantly, not always matching the regex!

I'm not filtering at the item level, so ALL events are received, and wish to send an alert when an event comes in with matching text containing a number between 0 and 4999.

I *think* its because events are coming in too quickly, so if you get say the following, at the same time...

The table "OrderHistory Store" has 278 MB of free space
The table "OrderHistory Store" has 5139 MB of free space

... it is alerting only the last one, and the first does not show up :/

it also explains why I received an alert for "eseutil (25464) The database engine (8.03.0083.0000) started a new instance (0)." in the previous email, as this was the next event in windows event viewer, immediately preceding it were two others like above, one which was a low enough value.

say you match on simply "ABC [0-9]{1,3}" i.e. ABC 100 will trigger..

at midnight 4 events are produced, that occur between item refresh

00:00:01 ABC 443
00:00:01 ABC 244
00:00:01 ABC 6988
00:00:03 DEFGHI

the emailed alert will contain DEFGHI instead of two alerts ABC 443 and ABC 244 which is clearly wrong

I think this will be repeatable - if you can generate events quickly enough.

The agent has the default configuration for refresh active checks (2 minutes apparently), in zabbix server, the item is set to every 30 seconds.

How do we fix this fairly nasty problem? is this fixable by configuration or is it a bug?

Keith

[zalex_ua]

Still having reliability issues with EventLog regex monitoring, triggers are firing fairly consistently but not reliably, and importantly, not always matching the regex!

Fresh issue registered. Mayde some problems related to this issue https://support.zabbix.com/browse/ZBX-3322

I *think* its because events are coming in too quickly, so if you get say the following, at the same time...

Read this issue https://support.zabbix.com/browse/ZBXNEXT-457 and all registered issues witch related to "nanoseconds"
Witch macro you are used for email? I hope the {ITEM.VALUE}

at midnight 4 events are produced, that occur between item refresh
.... in zabbix server, the item is set to every 30 seconds.

I can not recommend this, but for 'eventlog' keys I use item refresh interval - 1 seconds.
It does not prevent all problems, but I want to have the agent sent events as quickly as possible (and suddenly windows die faster than an agent will have time to read a eventlog? )
Do not forget that the agent is still a buffer of time - 5 seconds by default.

the emailed alert will contain DEFGHI instead of two alerts ABC 443 and ABC 244 which is clearly wrong

Read my previous thoughts about macro {ITEM.VALUE} and about nanoseconds precision.

How do we fix this fairly nasty problem? is this fixable by configuration or is it a bug?

Believe me - I went through it all, and all these problems are solved.
I see only one solution - Zabbix 2.0
I have long ago been using trunk (alfa version for 2.0), where these functions are implemented. But for Production is not recommended. If you ready to hard way - let's try

[kehall]

Fresh issue registered. Mayde some problems related to this issue https://support.zabbix.com/browse/ZBX-3322

Yes I think that probably refers to the original issue of having to put the regex in global.

Read this issue https://support.zabbix.com/browse/ZBXNEXT-457 and all registered issues witch related to "nanoseconds"
Witch macro you are used for email? I hope the {ITEM.VALUE}

Ahh ok, stupidly I had LASTVALUE in there, which yep, makes sense.

I can not recommend this, but for 'eventlog' keys I use item refresh interval - 1 seconds.

I'll try this to see if I can help the multiple events issue but we do receive events with the same timestamp basically, and need alerting to each one, having hopefully fixed the VALUE issue we'll monitor this..

It does not prevent all problems, but I want to have the agent sent events as quickly as possible (and suddenly windows die faster than an agent will have time to read a eventlog? )
Do not forget that the agent is still a buffer of time - 5 seconds by default.

Just a thought, is there no mechanism in windows to actually hook into event generation and send them to zabbix as they are produced (rather than relying on reading them)?

Believe me - I went through it all, and all these problems are solved.
I see only one solution - Zabbix 2.0
I have long ago been using trunk (alfa version for 2.0), where these functions are implemented. But for Production is not recommended. If you ready to hard way - let's try

Since this is production I think 2.0 may be a bit premature

Thanks

Keith

[zalex_ua]

Just a thought, is there no mechanism in windows to actually hook into event generation and send them to zabbix as they are produced (rather than relying on reading them)?

Keith

Yes. Se my worked solution in attachment.
Unfortunately I do not talk in detail that there is - if you like - will deal independently.
This is solution works absolutely without Zabbix !