windows event log processing

[swaterhouse]

I have been chasing some very strange "network" issues in the last couple of weeks. One of the things I found was that a couple of my slow\overloaded servers were considerably worse in the last couple of weeks than they have been in the past. I finally narrowed it down to zabbix event log monitoring in 1.4.2. Up until 3 weeks ago I was using version 1.1.6 of the agent and never really had any issues with it.

Attached is a graph of cpu load and util showing a 48 hour time span. I stopped monitoring the event logs at about 9:30 AM yesterday (which is just about the center of each graph) and you can clearly see a massive drop in the number of cpu load and utilization spikes. I saw the same results on three other servers that I tested this on. This only seems to be an issue with servers that are CPU bound (i.e. they don't have enough cpu power available) but you can see a "spike" in activity on all servers that do event log monitoring.

Anyone else see this? Were there any changes to event log processing that hurt the performance that badly?

Edit:
I also restarted the agents on one of the servers I tested as well and in the 24 hour period when I was doing event logs checks the zabbix_agentd.exe process had used about 34 minutes of cpu time. In the 24 hour period since I stopped doing eventlog checks the zabbix_agentd.exe process has only used 28 seconds of CPU time.

[swaterhouse]

Is anyone else seeing anything like this?

Two servers running all the same checks except one doesnt do event logs the other does. Both restarted 3 days ago.

CPU Time for zabbix_agentd.exe (taken from task manager)
server with no event log checks 00:01:36 (1 minute 36 seconds)
server with event log checks 00:34:22 (32 minutes 22 seconds)

[bbrendon]

I've seen other interesting things with event logs, but not what you describe. All the windows servers I monitor aren't very busy. Only the UNIX ones.